AD  All  8721 


AFHRL-TR-82-6 


AIR  FORCE 


VISION  RESEARCH  FOR  FLIGHT  SIMULATION 


Edited  by 

Whitman  Richards 
Key  Distnukeg 

National  Research  Council 
Assembly  of  Behavioral  and  Social  Sciences 
2101  Constitution  Avenue 
Washington,  D.C.  20418 


OPERATIONS  TRAINING  DIVISION 
Wiliiams  Air  Force  Base,  Arizona  85224 


July  1982 

Final  Report 


Approved  for  public  release;  distribution  unlimi 


imiled. 


LABORATORY 


AIR  FORCE  SYSTEMS  COMMAND 

BROOKS  AIR  FORCE  BASEJEXAS  78235 

82 


NOTICE 


Wlipii  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose 
oilier  than  in  connection  with  a  definitely  Government-related  procurement,  the  I  nited 
Slates  Government  incurs  no  responsibility  or  any  obligation  whatsoever.  The  fact  that 
the  Government  may  have  formulated  or  in  any  way  supplied  the  said  drawings, 
specifications,  or  other  data,  is  not  to  be  regarded  by  implication,  or  otherwise  in  any 
manner  construed,  as  licensing  (he  holder,  or  any  other  person  or  corporation:  or  as 
conveying  any  rights  or  permission  to  manufacture,  use.  or  sell  any  patented  invention 
that  may  in  any  way  be  related  thereto. 

The  Public  Affairs  Office  has  reviewed  this  report,  and  it  is  releasable  to  the  National 
Technical  Information  Service,  where  it  will  be  available  to  the  general  public, 
including  foreign  nationals. 

This  report  has  been  reviewed  and  is  approved  for  publication. 

THOMAS  M.  LONGRIOGE 
Contract  Monitor 


MILTON  E.  WOOD.  Technical  Director 
Operations  Training  Division 

RONALD  W.  TERRY.  Colonel,  (  SAP 
Commander 


1 


[  ndassified 


SECURITY  CLASSIFICATION  OF  THIS  PAGE  (When  Data Entered) 


|  REPORT  DOCUMENTATION  PAGE 

READ  INSTRUCTIONS 

BEFORE  COMPLETING  FORM 

1.  REPORT  NUMBER 

2  GOVT  ACCESSION  NO. 

3  RECIPIENT'S  CATALOG  NUMBER 

AFHRL-TR-82-6 

4-  title  (and  Subtitle) 

5.  TYPE  OF  REPORT  A  PERIOD  COVERED 

VISION  RESEARCH  FOR  FLIGHT  SIMLLATION 

Final 

6.  PERFORMING  O^G.  REPORT  NUMBER 

7.  author^*; 

6.  CONTRACT  OR  GRANT  NUMBERf*) 

Whitman  Richards 

Key  Dismukes 

F33615-80-C-(HMW 

9  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

National  Research  Council 

Assembly  of  Behavioral  and  Social  Sciences 

2101  Constitution  Avenue 

Washington.  D.C.  20418 

10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  A  WORK  UNIT  NUMBERS 

61I01F 

iLMbKit; 

tl.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

12.  REPORT  DATE 

July  1982 

Brooks  Air  Force  Base.  Texas  78235 

13.  NUMBER  OF  PAGES 

108 

1  14.  MONITORING  AGENCY  NAME  ft  ADDRESS^//  different  from  Controlling  Office) 

15.  SECURITY  CLASS,  (of  this  report) 

Operations  Training  Division 

Air  Force  Human  Resources  Laboratory 

l  Classified 

W  illiams  Air  Force  Base.  Arizona  85224 

15a.  DECLASSIFICATION  DOWNGRADING 
SCHEDULE 

ts.  DISTRIBUTION  STATEMENT  (of  this  Report) 

Approved  for  public  release:  distribution  unlimited. 

17.  DISTRIBUTION  STATEMENT  (of  the  abstract  entered  in  Btock  20,  ft  different  from  Report) 

16.  SUPPLEMENTARY  NOTES 

19.  KEY  WORDS  (Continue  on  reverse  aide  ft  neceaaary  and  Identify  by  block  number) 

flight  simulation  visual  cues 

low-level  flight  visual  displays 

vlslon  visual  information  processing 

20,  JkBSTRACT  (Continue  on  reverse  aids  It  neceaaary  and  Identify  by  block  number) 

This  report  is  based  on  a  workshop  organized  by  the  Committee  on  Vision  of  t lie  National  Researelt  Council  and 
by  the  Operations  Training  Division  of  the  Air  For<e  Human  Resources  Laboratory  at  Williams  AFB  in  June  1980. 

The  workshop  brought  together  vision  scientists  from  academia  and  government  scientists  concerned  with  research  on 
visual  displays  for  flight  simulation.  The  principal  objective  was  to  provide  recommendations  concerning  fruitful 
approaches  for  the  conduct  of  research  on  what  visual  information  is  needed  for  simulation  and  how  it  might  best  be 
presented.  Low-level  flight  was  used  as  a  focus  for  discussion  of  problem-solving  approaches.  The  technical  report 
prepared  by  the  steering  group  provides  examples  of  particular  research  strategies  that  might  help  elucidate  several  of 
the  long-range  issues  in  visual  simulation.  Several  strategies  are  suggested  for  exploring  what  visual  features  may  he 
used  in  low-level  flight:  systematic  contribution  expert  opinions,  geometric  analysis  of  terrain  information,  and 

DD  ,  j2SM73  1473  EDITION  Of  1  NOV  *3  IS  OBSOLETE  I  m  Uxifi.-d 


SECURITY  CLASSIFICATION  OF  THIS  PACE  (Wh*n  D urn  En(.r.d) 


SECURITY  CLASSIFICATION  of  PAOE(TFh»n  Data  Enlaratf) 


SUMMARY 


Objective* 

This  effort  had  two  objectives.  The  first  objective  was  to  promote  interaction  between  vision-research 
scientists  and  personnel  working  on  flight  simulator  research  and  development  (R&D).  The  second  objective  was  a 
set  of  recommendations  regarding  R&D  needs  and  strategies  that  might  be  useful  in  developing  more  effective 
displays  of  visual  scenes  in  flight  simulation,  particularly  for  low-level  flight. 

Background 

In  general,  the  development  and  use  of  visual  displays  in  flight  simulation  have  been  dominated  by 
engineering  concerns.  Equipment  for  simulating  visual  scenes  has  become  highly  sophisticated,  but  vision  science 
has  played  a  limited  role  in  determining  what  visual  information  is  presented  in  simulators  and  how.  As  a  result, 
fundamental  questions  remain  unanswered  as  to  how  to  make  the  visual  displays  used  in  flight  simulators  most 
effeciive  or  appropriate  for  flight  training. 

Approach 

A  workshop  was  organized  by  the  Committee  on  Vision  of  the  National  Research  Council  and  the  Operations 
Training  Division  of  the  Air  Force  Human  Resources  Laboratory  (AFHRL/OT),  Williams  AFB,  Arizona.  This 
workshop  was  held  in  June  1980  at  Williams  AFB  and  was  attended  by  vision  scientists  from  the  academic 
community,  as  well  as  personnel  from  industry  and  the  military  services  who  are  engaged  in  R&D  on  visual 
displays  for  flight  simulation.  Following  the  workshop,  members  of  the  Committee  on  Vision  identified  key  R&D 
issues  and  strategies. 

Specifics 

The  two-day  workshop  began  with  presentations  to  familiarize  participants  with  the  visual  simulation  R&D  at 
AFHRL/OT.  This  was  followed  by  a  tour  of  the  wide-field-of-view  computer  image  generation  facility  there.  A 
second  set  of  presentations  informed  the  AFHRL/OT  staff  about  complementary  research  activities  in  portions  of 
the  academic  community. 

A  steering  group  then  developed  examples  of  specific  R&D  approaches  that  might  help  clarify  several  of  the 
long-range  issues  confronted  in  visual  simulation.  Primary  emphasis  was  given  to  low-level  flight,  in  which  the 
extraction  of  visual  information  from  terrain  features  is  crucial  but  not  well  understood.  Several  strategies  were 
suggested  for  exploring  what  visual  features  may  be  used  in  low-level  flight.  Specifically  discussed  were:  (a) 
systematic  condensation  of  opinions,  particularly  those  of  pilots,  (b)  geometric  analysis  of  terrain  information,  and 
(c)  psychophysical  analyses  of  visual  processing  modalities. 


Concluskms/Recommendations 

The  workshop  format  promoted  productive  technical  interactions  among  the  participants.  It  was  concluded 
that  multiple  R&D  lines  were  needed  to  answer  the  fundamental  questions  about  visual  display  requirements  for 
flight  simulation.  A  mixture  of  R&D  approaches  was  recommended,  ranging  from  short-term  attacks  on  issues  of 
immediate  operational  concerp,  to  long-range  investigations  of  fundamental  issues.  Need  for  greater  attention  to 
training  issues  was  noted,  since  in  many  cases  the  best  R&D  approach  to  visual  problems  cannot  be  determined 
until  training  needs  are  specified. 
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PREFACE 


T!i<‘  work  reported  herein  was  accomplished  in  support  of  the  Air  Force  Human  Resources  Laboratory's 
Air  Combat  Tactics  and  Training  Research  Thrust.  In  order  to  effectively  utilize  flight  simulators  to  support 
advanced  air  combat  training  requirements,  significant  improvements  in  the  display  of  out-of-cockpit  flight 
environments  will  be  required.  The  purpose  of  this  work  unit  was  to  focus  existing  expertise  in  visual  science 
on  the  design  of  research  strategies  appropriate  for  resolution  of  major  issues  in  visual  simulation 
requirements. 

This  report  is  based  on  a  workshop  held  in  June  1980  to  discuss  problems  encountered  in  visual 
simulation  of  flight  and  to  explore  ways  of  stimulating  belter  research  in  this  area.  Low-level  flight  was 
chosen  as  a  problem  example  for  discussion  of  how  various  research  strategies  might  be  applied.  This  limited 
effort  is  not  intended  to  be  a  comprehensive  analysis  of  the  problems  encountered  in  visual  simulation  of 
flight,  and  there  has  been  no  attempt  to  lay  out  priorities  for  research.  Such  a  comprehensive  assessment 
would  be  an  enormous  undertaking  requiring  a  much  more  broadly  constituted  group.  Rather,  the  approach 
has  been  to  attempt  to  give  a  sense  of  the  complicated  mixture  of  issues  encountered  in  determining  what 
visual  display  information  is  required  for  effective  simulation  of  a  given  flight  environment.  Examples  are 
presented  to  show  how  basic  research  perspectives  might  be  used  to  attack  these  issues. 

Individual  research  perspectives  of  some  members  of  the  steering  group  are  included  as  appendixes  to 
this  report.  The  body  of  the  report  discusses  general  issues,  suggests  ways  to  facilitate  interaction  of  basic 
scientists  and  simulation  engineers,  and  attempts  to  show  the  complementarity  of  several  approaches  to 
research.  The  entire  group  discussed  the  body  of  the  report,  written  by  Whitman  Richards  with  the  assistance 
of  Key  Dis. nukes. 

Robert  Hennessy  of  the  National  Research  Council  and  Stanley  Collyer  of  the  Naval  Training  Equipment 
Center  substantially  assisted  the  steering  group  with  numerous  discussions,  suggestions,  and  identification  of 
relevant  literature.  With  some  effort,  they  provided  the  steering  group  a  quick  education  in  'lie  practical 
problems  of  simulation.  George  Ruckland.  the  Air  Force  project  officer  for  this  study,  played  a  major  role  in 
organizing  the  workshop  on  which  this  report  is  based.  Lloyd  Kaufman  and  Conrad  Kraft  made  helpful 
comments  on  an  early  draft  e  his  report  and  provided  thoughtful  suggestions  about  the  problems  of  visual 
si,..,,  jtiou  research. 

The  project  that  is  the  subject  of  this  report  was  approved  by  the  Governing  Board  of  the  National 
Research  Council,  whose  members  are  drawn  from  the  councils  of  the  National  Academy  of  Sciences,  the 
National  Academy  of  Engineering,  and  the  Institute  of  Medicine.  The  members  of  the  committee  responsible 
for  the  report  were  chosen  for  their  special  competences  and  with  regard  for  appropriate  balance. 

This  report  has  been  reviewed  by  a  group  other  than  the  authors  according  to  procedures  approved  by  a 
Report  Review  Committee  consisting  of  members  of  the  National  Academy  of  Sciences,  the  National  Academy 
of  Engineering,  and  the  Institute  of  Medicine. 

The  National  Research  Council  was  established  by  the  National  Academy  of  Sciences  in  1916  to  associate 
the  broad  community  of  science  and  technology  with  the  Academy's  purposes  of  furthering  knowledge  and 
of  advising  the  federal  government.  The  Council  operates  in  accordance  with  general  policies  determined  by 
the  Academy  under  the  authority  of  its  congressional  charter  of  1863.  which  establishes  the  Academy  as  a 
private,  nonprofit,  self-governing  membership  corporation.  The  Council  has  become  the  principal  operating 
agency  of  both  the  National  Academy  of  Sciences  and  the  National  Academy  of  Engineering  in  the  conduct  of 
their  services  to  the  government,  the  public,  and  the  scientific  and  engineering  communities.  It  is 
administered  jointly  by  both  Academies  and  the  Institute  of  Medicine.  The  National  Academy  of  Engineering 
and  the  Institute  of  Medicine  were  established  in  1*164  and  1970  respectively,  under  the  charter  of  the 
National  Academy  of  Sciences. 
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VISION  RESEARCH  FOR  FLIGHT  SIMULATION 


INTRODUCTION 


The  U.S.  Air  Force  Human  Resources  Laboratory  (USAFHRL)  asked  the 
Committee  on  Vision  to  organize  a  workshop  on  vision  research  issues  in 
flight-training  simulators,  particularly  those  using  computer  image 
generation  (CIG)  techniques.  The  workshop,  held  at  Williams  Air  Force 
Base,  Ariz . ,  in  June  1980,  had  two  purposes.  First,  it  was  an  experi¬ 
ment  in  facilitating  interaction  between  vision  scientists  and  person¬ 
nel  working  on  flight  simulator  research.  Second,  it  explored  research 
needs  and  strategies  that  might  be  useful  in  developing  more  effective 
display  of  visual  scenes,  particularly  for  low-level  flight.1 

In  general,  the  development  and  use  of  visual  displays  in  flight 
simulation  have  been  dominated  by  engineering  concerns.  Equipment  for 
simulating  visual  scenes  has  become  highly  sophisticated,  but  vision 
science  has  played  a  small  role  in  determining  what  visual  information 
is  presented  in  simulators  and  the  way  it  is  presented.  Vision  scien¬ 
tists  have  done  limited  work  on  determining  the  information  required 
for  effective  visual  simulation.  Well-controlled  experiments  to  com¬ 
pare  training  effectiveness  of  different  approaches  to  visual  simula¬ 
tion  are  expensive,  difficult,  and  time  consuming,  partly  because  they 
cannot  be  readily  conducted  in  laboratories.  Little  theoretical  work 
is  available  to  predict  the  kinds  of  visual  information  that  should  be 
included  in  a  particular  simulation.  Consequently,  simulator  designers 
and  buyers  have  typically  opted  for  as  much  "realism"^  and  display  capa¬ 
bility  as  possible,  since  the  research  literature  does  not  provide  a 
basis  for  selecting  or  evaluating  simpler  displays. 

The  two-day  workshop  (see  appendix  F  for  topics  and  participants) 
began  with  presentations  to  familiarize  participants  with  visual  simu¬ 
lation  research  efforts  at  the  Operations  Training  Division  of  USAFHRL 
at  Williams  AFB  and  was  followed  by  a  demonstration  of  the  wide-field 
CIG  used  for  research  there.  A  second  set  of  presentations  informed 
the  Williams  staff  about  complementary  research  activities  in  portions 


■'‘Low-level  flight  is  defined  as  flying  along  the  contour  of  the  earth, 
usually  at  an  altitude  below  200  feet  (above  ground  level). 
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Realism  is  used  here  to  include  both  accuracy  (fidelity  to  real-world 
characteristics)  and  completeness  of  detail. 
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of  the  academic  community.  During  the  first  day,  it  rapidly  became 
obvious  that  an  enormous  number  of  research  issues  are  involved  in  at¬ 
tempting  to  develop  effective  visual  simulations.  These  issues  in¬ 
clude  CIG  techniques,  the  capabilities  of  the  human  visual  system,  in¬ 
formation  processing  needs,  attentional  factors,  measures  of  training 
effectiveness,  and  strategies  for  enhancing  training  (see  table  1). 

But  researchers  have  made  little  progress  in  isolating  components  of 
major  simulation  issues  so  that  they  can  be  attacked  separately.  Also, 
training  needs  apparently  have  not  been  sufficiently  well  character¬ 
ized  to  allow  visual  requirements  to  be  defined  accordingly.  This 
range  of  problems  was  clearly  too  large  for  a  small  group  of  vision 
scientists  to  address  in  depth  in  a  single  workshop.  Thus,  on  the 
second  day,  the  discussion  concentrated  on  possible  research  strategies 
for  determining  what  visual  information  is  needed  in  a  simulator  for  a 
pilot  to  perform  a  required  flight  task.  The  issues  of  training  and 
simulator  evaluation  were  set  aside,  although  participants  continually 
noted  that  these  issues  are  important  and  deserve  careful  consideration 
(see  next  section) . 

Visual  scene  requirements  differ  considerably  among  flight  tasks 
simulated,  and  research  needs  will  vary  accordingly.  A  particular 
flight  environment,  low-level  flight,  was  chosen  as  a  problem-solving 
example  for  the  second  day  of  workshop  discussion.  This  choice  was 
based  on  several  factors.  Acquisition  of  basic  flying  skills,  such  as 
take-off  and  landing,  has  been  shown  to  occur  with  simulation  training, 
even  with  fairly  simple  visual  displays  (Semple  et  al . ,  1980;  Waag, 

1981) .  Much  less  is  known  about  the  visual  requirements  for  simulating 
advanced  tasks  such  as  low-level  flight  for  either  initial  acquisition 
or  maintenance  of  skills.  Low-level  flight  is  an  important  Air  Force 
mission,  yet  it  has  been  difficult  to  simulate  adequately  with  CIG  sys¬ 
tems.  Even  experienced  pilots  may  misjudge  altitude,  range,  and  shape 
of  the  ground  in  simulators,  problems  that  suggest  that  some  unknown 
aspects  of  the  visual  display  are  inadequate  to  allow  performance 3  com¬ 
parable  to  that  in  the  real  world.  Low-level  flight  may  be  a  worst- 
case  example  in  which  visual  demands  are  great  and  simulation  require¬ 
ments  are  not  understood.  Research  approaches  that  help  determine 
visual  display  requirements  for  low-level  flight  may  also  prove  useful 
in  analyzing  visual  requirements  for  other,  simpler  flight  tasks. 

After  the  workshop,  the  steering  group  met  several  times  to  discuss 
research  issues  and  prepare  this  report.  We  do  not  intend  this  limited 
effort  to  be  a  comprehensive  analysis  of  the  problems  encountered  in 
visual  simulation  of  flight,  and  we  have  not  attempted  to  establish  re¬ 
search  priorities.  Such  a  comprehensive  assessment  would  be  an  enor¬ 
mous  undertaking  requiring  a  much  more  broadly  constituted  group.  We 
have  attempted  to  give  a  sense  of  the  complicated  mixture  of  issues 
encountered  in  attempting  to  determine  the  visual  information  required 
for  effective  simulation  of  a  given  flight  environment.  This  report 
has  two  main  parts.  First,  we  will  discuss  the  general  need  for 


However,  see  discussion  in  the  following  section  of  whether  performance 
in  simulators  is  an  adequate  measure  of  training  effectiveness. 
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TABLE  1  Some  Issues  in  Simulator  Training  Research 


Training  &  Management 
Issues 

Inst  ructional 

Issues 

Hardware 

Requirements 

Tasks  for  Simulation 
Training 

Role  of  Instructor 

Visual  System 

Training  Strategies 

Task  Difficulty  Level 

Motion  Systems 

Initial  Training  vs. 

Augmentation  of  Feedback 

A/C  Dynamics 

Skill  Maintenance 

Scheduling  of  Feedback 

Instructional 

Features 

Whole  vs .  Part  Task 

Adaptive  vs.  Fixed 

Training 

Amount  of  Training 

Motion-Visual 

Synchronization 

Knowledge-Based 

Visual  &  Display 

Issues 

Behavioral  Issues 

Issues 

Experimental 

Methodologies 

Individual  Differences 

Image  Quality 

Performance 

Attention  and  Workload 

Image  Detail 

Assessment 

Skill  Degradation 

FOV  Size 

Application  of  Basic 
Research  Findings 

Motivation 

Components  of  Skills 

Perception 

Cognition 

Response  Processes 

Realism 

SOURCE:  R.  Hennessy ,  unpublished. 
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research  in  this  area  in  a  way  we  hope  will  convey  the  magnitude  of  the 
problems  encountered.  In  this  context,  we  will  suggest  ways  to  facili¬ 
tate  the  research  needed  and  to  stimulate  interaction  between  scien¬ 
tists  and  simulation  researchers.  Second,  we  will  describe  some  of  the 
technical  problems  in  more  detail,  giving  a  few  examples  of  applying 
basic  research  strategies  to  these  problems.  We  have  identified  some 
areas  of  research  that  may  be  relevant  to  the  simulation  problems;  how¬ 
ever,  these  examples  are  not  intended  as  a  comprehensive  list. 

Individual  authors  developed  the  examples  of  research  strategies 
as  appendix  papers  for  this  report.  We  asked  the  authors  to  describe, 
preferably  by  example,  the  value  and  pitfalls  of  the  particular  re¬ 
search  approaches  they  would  pursue.  These  diverse  approaches  are 
complementary;  we  will  attempt  to  show  in  the  body  of  the  report  how 
they  might  be  combined  to  address  some  aspects  of  the  problem  of  simu¬ 
lating  low-level  flight.  Some  authors  developed  their  approaches  from 
a  particular  theoretical  perspective;  we  will  briefly  discuss  assump¬ 
tions  underlying  these  perspectives,  but  we  will  not  debate  whether 
they  will  eventually  be  proven  correct.  At  this  point  in  simulation 
research,  the  value  of  such  theoretical  orientations  is  in  helping 
formulate  sharp  questions  that  will  generate  useful  data. 

Other  examples  of  research  approaches  could  have  been  chosen 
equally  well  for  these  illustrations.  There  is  no  one  best  research 
approach  to  a  field  as  large  and  uncertain  as  visual  simulation,  even 
if  one  concentrates  on  a  particular  task  simulation.  Rather,  a  diverse 
mixture  of  research  approaches  should  be  encouraged.  Indeed,  the  mem¬ 
bers  of  this  small  steering  group  represent  a  range  of  research  per¬ 
spectives,  and  each  would  attack  these  problems  differently.  The  choice 
of  these  research  examples  is  oriented  toward  the  particular  problems 
raised  by  low-level  flight.  For  instance,  several  examples  emphasize 
geometric  aspects  of  terrain.  Some  or  all  of  these  research  approaches 
might  also  be  used  to  study  simulation  of  other  flight  tasks;  however, 
the  emphasis  would  be  different.  We  stress  that  effective  research 
strategies  must  include  an  analysis  of  the  particular  features  and  re¬ 
quirements  of  the  flight  task  to  be  simulated. 

Simulation  workers  will  note  that  most  of  these  examples  of  re¬ 
search  strategy  differ  from  those  typically  found  in  simulation  re¬ 
search  literature.  In  contrast  to  simulation  research  focused  on  ex¬ 
isting  equipment,  these  examples  of  basic  research  address  long-range 
issues  of  visual  simulation.  We  have  emphasized  the  fundamental  side 
because  that  is  the  aspect  for  which  the  Air  Force  has  the  fewest  re¬ 
sources  and  in  which  we  are  best  able  to  assist. 
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BACKGROUND:  PROBLEMS  IN  VISUAL  SIMULATION  RESEARCH4 

Simulators  are  widely  used  for  commercial,  military,  and  space 
flight  training.  Simulation  training  has  been  shown  to  reduce  substan¬ 
tially  the  number  of  actual  flying  hours  required  for  student  pilots 
to  master  basic  skills  and  for  experienced  pilots  to  learn  about  new 
aircraft.  In  addition  to  reducing  training  costs,  simulators  can  be 
used  to  train  pilots  safely  in  hazardous  maneuvers  and  to  create  situ¬ 
ations  that  do  not  occur  or  cannot  be  adequately  controlled  in  the  real 
world  (Hennessy,  1981). 

Rapid  improvements  in  computer  image  generation  (CIG)  techniques 
in  the  past  decade  have  resulted  in  fairly  sophisticated  visual  dis¬ 
plays  for  simulation.  Nevertheless,  the  contribution  of  visual  dis¬ 
plays  to  overall  training  effectiveness  of  simulators  is  poorly  under¬ 
stood.  A  few  studies  have  systematically  examined  effects  of  visual 
display  features  on  training  effectiveness  (see,  for  example,  Semple 
et  al . ,  1980;  Waag,  1981).  Concern  about  visual  display  effectiveness 
is  being  raised,  particularly  among  military  R&D  personnel,  because  of 
several  developments,  especially  the  following: 

1.  The  costs  of  advanced  simulators  are  rising  rapidly.  Visual 
simulation  has  been  pushed  to  provide  all  the  realism  and 
capability  afforded  by  rapid  advances  in  technology,  with 
attendant  rises  in  cost. 

2.  Some  researchers  have  questioned  the  value  of  realism  as  a 
guide  for  visual  display  (see,  for  example,  Coblitz,  1980; 
Hennessy  et  al . ,  1980). 

3.  Simulation  of  advanced  flight  maneuvers  and  environments, 
such  as  low-level  flight,  has  been  found  to  be  problematic. 

The  adequacy  of  training  in  existing  simulators  for  low-level 
flight  and  other  visually  demanding  tasks  is  not  clear  from 
the  few  existing  studies. 

The  development  and  use  of  visual  displays  in  simulators  have  been 
dominated  by  engineering  considerations . 5  Visual  scientists  and  psy¬ 
chologists  have  played  only  a  minor  role  in  deciding  what  visual  in¬ 
formation  should  be  displayed  and  how.  Design  engineers  have  developed 
visual  displays  based  on  common  sense  and  previous  experience.  The 
major  criterion  of  adequacy  of  displays  has  been  their  degree  of  ac¬ 
ceptance  by  experienced  pilots.  The  evaluations  of  both  pilots  and  de¬ 
signers  appear  to  be  based  largely  on  how  realistic  visual  displays 
appear,  rather  than  explicit  consideration  of  training  effectiveness. 

The  contributions  of  visual  scientists  have  been  limited,  largely 
because  they  have  not  been  able  to  tell  designers  the  kind  of  informa¬ 
tion  that  visually  displayed  scenes  should  contain.  Our  knowledge  of 
higher-order  perceptual  processes  is  too  limited  to  specify  with 
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For  more  extensive  discussions,  see  Brown  (1976),  Hennessy  et  al. 
(1980),  and  Semple  et  al.  (1980). 

^For  more  extensive  discussion  of  this  issue,  see  Boff  and  Martin 
(1980),  Fulgham  (1978),  and  Hennessy  (1981). 
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certainty  the  forms  in  which  visual  information  is  extracted  by  pilots 
performing  particular  tasks  in  the  real  world.  Uncertainty  about  both 
visual  information  processing  and  perceptua’  -md  cognitive  factors  in 
training  makes  it  difficult  to  devise  coherent  strategies  for  display 
of  visual  information  in  simulators.  So  far,  the  contribution  of  vision 
science  to  simulation  research  has  been  primarily  in  the  area  of  image 
quality  (resolution,  contrast,  distortion,  etc.).  Psychophysical  stud¬ 
ies  (e.g.,  Kraft  et  al . ,  1980)  of  human  visual  sensitivities  have  pro¬ 
vided  information  needed  by  engineers  attempting  to  design  displays  to 
match  human  capabilities. 

Semple  et  al .  (1980)  point  out  that  because  of  this  uncertainty 
about  visual  information  requirements,  it  has  probably  been  reasonable 
for  designers  to  emphasize  development  of  visual  displays  with  maximum 
detail  and  fidelity.  No  evidence,  however,  supports  the  common  assump¬ 
tion  that  increasing  realism  improves  the  training  value  of  simulators 
(see  also  Waag,  1981).  Furthermore,  highly  realistic  display  of  ex¬ 
tended  terrain  for  missions  such  as  low-level  flight  is  not  possible 
with  existing  CIG  equipment,  and  there  may  be  inherent  limitations  to 
display  realism  (see  appendix  B) . 

Training  issues  are  beyond  the  scope  of  this  report  and  the  compe¬ 
tence  of  its  authors;  however,  visual  display  issues  must  be  considered 
in  the  context  of  training  requirements.  Training  simulators  may  be 
used  for  acquisition  of  basic  flight  skills  by  student  pilots,  for 
transition  to  other  types  of  aircraft  by  experienced  pilots,  and  for 
proficiency  maintenance  in  a  given  aircraft.  Visual  display  require¬ 
ments  may  differ  considerably  among  these  three  training  roles  and  with 
different  flight  tasks.  For  example,  acquisition  of  basic  skills  has 
been  shown  to  be  substantially  enhanced  by  simulation  training  with 
simple  and  in  some  cases  highly  unrealistic  visual  displays  (Hennessy 
et  al.,  1981).  In  contrast,  proficiency  maintenance  of  visually  de¬ 
manding  tasks  might  require  more  detailed  visual  presentation;  however, 
in  the  absence  of  data  we  can  only  speculate  on  this  point. 

Collyer  has  suggested  (private  communication)  that  three  kinds  of 
skills  are  acquired  in  flight  training:^ 

1.  Perceptual:  "When  the  world  looks  like  this,  what  is  the 
aircraft's  situation?" 

2.  Decision/procedural:  "When  the  aircraft  is  in  this  situation, 
what  needs  to  be  done  next?" 

3.  Control:  "How  do  I  make  it  do  that?” 

Traditionally,  flight  instruction  has  emphasized  procedural  and  control 
skills;  little  attention  has  been  given  to  perceptual  skills,  and  in 
fact  little  is  known  about  perceptual  learning  processes  in  flight 
training.  It  may  be  that  the  demonstrated  effectiveness  of  simulators 
in  training  basic  flight  skills  results  mainly  from  learning  procedural 
and  control  skills,  and  in  this  case  the  degree  of  realism  of  the 
visual  display  may  not  be  very  important.  Perceptual  learning  might 
play  a  large'-  role  in  learning  and/or  maintaining  proficiency  of  some 
flight  tasks  than  in  others.  Resolution  of  this  issue  would  facilitate 


6 


Also  see  Roscoe  (1979)  on  this  point. 
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cost-effective  design  of  simulators  for  specific  tasks.  Unfortunately, 
there  has  been  little  good  research  on  this  point,  although  some  recent 
studies  appear  promising  (e.g..  Coblitz,  1980;  Hennessy  et  al.,  1981). 

The  ultimate  measure  of  simulator  training  effectiveness  is  the 
enhancement  of  actual  flight  performance.  Unfortunately,  transfer-of- 
training  studies  are  difficult  and  costly,  and  they  must  compete  with 
operational  missions  for  pilots  and  equipment.  Although  transfer-of- 
tralning  studies  have  demonstrated  effectiveness  of  simulator  training 
for  a  number  of  casks,  almost  none  of  these  studies  has  been  designed 
to  reveal  the  contribution  to  training  transfer  of  the  different  fea¬ 
tures  of  the  equipment  and  simulation  instruction  (Waag,  1981).  Thus 
we  have  virtually  no  empirical  information  on  how  transfer  of  simula¬ 
tion  training  is  affected  by  visual  display  characteristics  or  by  the 
information  content  of  the  visual  scene.  Several  investigators  have 
suggested  that  Instruction  methods  used  in  simulator  training  may  have 
as  much  or  more  Impact  on  effectiveness  as  do  equipment  features  (e.g., 
Caro,  1977). 

Transfer-of-training  studies  that  simultaneously  evaluate  effects 
of  several  simulator  equipment  features,  trainees'  characteristics,  and 
training  procedure  variables  are  urgently  needed  (Simon  and  Roscoe, 

1981),  It  would  also  help  to  have  techniques  for  predicting  simulator 
training  effectiveness,  to  be  used  to  determine  which  issues,  procedures, 
and  equipment  warrant  full-scale  transfer  studies.  Pilots'  performance 
in  the  simulator  has  been  used  as  such  a  measure  (e.g.,  Westra  et  al., 
1981).  This  measure,  however,  is  problematic.  Factors  that  affect 
performance  in  the  simulator  may  not  have  the  same  effect  on  transfer 
of  training.  Platform  motion,  for  instance,  in  some  studies  improved 
pilot  performance  in  the  simulator  but  did  not  affect  transfer  to  actual 
flight  performance  in  these  and  other  studies  (e.g.,  Waag,  1981).  It 
is  sometimes  assumed  that  factors  that  do  not  Influence  performance  in 
the  simulator  will  not  contribute  to  transfer  of  training;  however,  this 
assumption  needs  careful  analysis  and  experimental  evaluation. 

A  major  problem  with  most  transfer-of-training  and  performance 
studies  is  that  their  designs  do  not  allow  generalizations  to  be  drawn. 

In  addition  to  not  discriminating  among  effects  of  different  simula¬ 
tion  factors,  most  studies  have  too  narrow  a  focus  (e.g.,  studies  on  a 
particular  piece  of  equipment  or  a  specific  operational  need)  to  shed 
light  on  the  general  questions  about  visual  information  requirements. 
These  studies  are  not  additive.  In  contrast,  the  work  of  Westra  and 
coworkers  (1981)  and  Simon  and  Roscoe  (1981),  among  others,  suggests 
possibilities  for  developing  more  powerful  empirical  methods  of 
evaluation. 

In  summary,  evaluation  studies  have  demonstrated  that  visual  simu¬ 
lation  can  substantially  enhance  training  of  some  (but  not  necessarily 
all)  flight  tasks.  There  is,  however,  little  information  on  the  rela¬ 
tive  training  effectiveness  of  different  approaches  to  visual  informa¬ 
tion  display.  Virtually  nothing  can  be  said  with  certainty  about  the 
best  way  to  display  visual  information  for  simulator  training  of  par¬ 
ticular  tasks. 
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RESEARCH  NEEDS 

As  pointed  out  earlier,  empirical  studies  of  the  relative  training 
effectiveness  of  different  approaches  to  visual  simulation  have  been 
quite  limited.  On  the  basis  of  either  existing  evaluation  studies  or 
theories  of  visual  perception,  it  is  impossible  to  specify  with  any 
certainty  the  visual  information  requirements  for  simulation  training 
of  specific  flight  tasks.  Simulators  in  use  today  provide  effective 
training  of  some,  not  all,  flight  tasks;  however,  it  is  important  to 
consider  cost  effectiveness  as  well  as  training  efficacy  (Orlansky  and 
String,  1977).  Lack  of  knowledge  of  visual  information  display  require¬ 
ments  makes  it  difficult  to  maximize  training  effectiveness  or  minimize 
costs. 

Research  on  visual  information  requirements  for  simulation  train¬ 
ing  has  lagged  for  several  reasons.  The  difficulty  and  cost  of  perform¬ 
ing  transfer-of-training  studies  and  the  incompleteness  of  knowledge  of 
visual  perception  have  been  mentioned  earlier.  Little  good  theoretical 
work  has  addressed  visual  simulation  problems.  For  instance,  little 
progress  has  been  made  in  understanding  geometric  aspects  of  information 
extraction  from  terrain  viewed  by  pilots  since  Carel's  (1961)  research. 
Work  is  limited  in  this  field  in  part  because  few  CIG  simulators  are 
available  for  research.  Recently,  researchers  have  used  two  simulators 
with  modem  visual  systems:  the  Advanced  Simulator  for  Pilot  Training 
(ASPT)  at  Williams  Air  Force  Base,  Ariz.,  and  the  Visual  Technology  Re¬ 
search  Simulator  (VTRS)  at  the  Naval  Training  Equipment  Center  in  Or¬ 
lando,  Fla.  Personnel  at  these  two  facilities  have  made  notable  efforts 
to  deal  with  some  of  the  research  issues  described  herein,  but  clearlv 
a  much  larger  enterprise  is  required  to  resolve  these  problems. 

A  comprehensive  analysis  of  research  needs  and  priorities  in  this 
field  is  beyond  the  scope  of  this  limited  study7;  however,  some  general 
observations  may  be  useful.  No  one  line  of  research  can  answer  the 
questions  raised  in  this  report.  A  mixture  of  research  approaches  is  ‘ 
needed,  ranging  from  short-term  attacks  on  issues  on  which  operational 
decisions  cannot  be  delayed  to  long-term  studies  of  fundamental  issues 
of  information  extraction  from  terrain  features.  Research  is  required 
for  both  visual  information  requirements  and  training  issues.  Much 
attention  should  be  given  to  designing  research  methods  so  that  knowl¬ 
edge  gained  from  diverse  studies  will  be  additive  instead  of  applicable 
only  to  the  particular  simulation  and  equipment  used  in  each  study. 

The  broad  issues  of  visual  simulation  requirements  need  to  be 
broken  into  discrete  components  that  are  potentially  solvable.  This 
task  alone  will  be  no  Bmall  feat.  Current  research  attempting  to  de¬ 
fine  relationships  between  visual  simulation  variables  and  flight  con¬ 
trol  performance  has  imnediate  value  for  evaluating  specific  displays 
and  equipment  features,  but  it  will  not  provide  fundamental  knowledge 
that  is  cumulative  and  that  might  allow  prediction  of  visual  informa¬ 
tion  requirements  for  a  wide  range  of  simulation  training  tasks.  Vision 
scientists  in  academia  may  be  able  to  help  devise  improved  research 
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See  Hennessy  et  al .  (1980)  for  one  such  effort. 
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approaches.  This  effort  will  require,  however,  long-term  commitment  of 
these  scientists  to  simulation  issues  and  availability  of  facilities  for 
such  work. 

Effective  research  on  visual  simulation  requires  work  in  both 
laboratory  settings  and  simulator  facilities.  In  both  settings,  more 
interaction  between  basic  scientists  and  simulation  engineers  is  needed. 
Most  vision  scientists  in  academia  are  not  familiar  enough  with  flight 
simulation  problems  to  be  aware  of  how  their  research  might  be  applied. 
Simulation  studies  appear  mainly  as  technical  reports,  rather  than  in 
the  journals  widely  read  by  vision  scientists,  and  are  relatively  in¬ 
accessible  to  the  academic  community.  Furthermore,  few  meetings  are 
held  for  simulation  engineers  and  vision  scientists  to  exchange  infor¬ 
mation  and  ideas.  The  human  factors  area,  in  which  such  interchange  is 
fairly  common,  might  be  a  good  model  for  interaction  of  vision  science 
and  simulation  study. 

To  improve  and  stimulate  information  exchange  between  the  academic 
and  simulation  communities,  the  steering  group  suggests  increasing  use 
of  three  kinds  of  mechanisms: 

1.  Conferences :  The  recent  Image  II  conference  illustrates  the 
value  of  forums  for  interaction  of  vision  scientists  and  simulation 
engineers.*  Direct  person-to-person  confrontation  is  the  most  rapid  and 
effective  way  to  become  aware  of  and  understand  the  pressing  issues  of 
the  moment.  (As  an  aside,  state-of-the-art  tutorials  in  relevant  areas 
of  science  might  be  considered  for  part  of  these  conferences.)  Such 
meetings  need  to  be  held  regularly  to  ensure  the  continuing  involvement 
of  che  scientific  community .  Also,  special  sessions  in  flight  simula¬ 
tion  could  be  encouraged  at  existing  meetings  of  scientific  societies. 

Such  sessions  have  the  potential  of  reaching  broad  segments  of  the  vision 
community.  The  annual  meetings  of  the  Human  Factors  Society  have  in¬ 
cluded  such  sessions  profitably,  and  a  session  on  visual  simulation  was 
held  at  the  October  1981  meeting  of  the  Optical  Society  of  America. 

Other  societies  that  might  be  asked  to  include  sessions  on  simulation 
include  IEEE,  the  Society  for  Information  Display,  and  the  Psychonomics 
Society.  National  Research  Council  symposia,  such  as  those  organized 

by  the  Committee  on  Vision  on  applied  visual  problems,  could  also  be 
helpful . 

2.  Proceedings  and  Review  Articles:  Proceedings  serve  the  obvi¬ 
ous  function  of  providing  information  about  the  content  of  a  meeting 
to  those  who  were  unable  to  attend.  Unfortunately,  proceedings  of 
simulation  conferences  have  not  always  been  widely  available.  Publish¬ 
ing  proceedings  in  a  journal  is  probably  the  most  satisfactory  way  to 
disseminate  this  information.  Some  consideration  also  should  be  given 
to  preparing  annual  reviews  of  vision  research  relevar"  to  simulation 
(for  engineers)  and  of  simulator  display  capabilities  ifor  vision 
researchers)  . 

3.  Exchanges :  Although  training  fellowships  exist  that  allow 
scientists  in  academia  to  visit  and  collaborate  at  simulator  facili¬ 
ties,  the  availability  of  such  fellowships  is  not  generally  known  in 
the  academic  community.  Greater  effort  should  also  be  made  to  provide 
opportunities  for  simulator  personnel  in  federal  agencies  to  visit  or 
study  in  university  laboratories.  Two  existing  mechanisms  that  should 

*  Monroe,  E.G.  1981  Image  II  Conference  Proceedings.  AFHRL-  TR-81-48 , 
AD-A110  226.  Williams  AFB,  AZ :  Operations  Training  Division,  Air  Force 
Human  Resources  Laboratory,  November  1981. 
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be  more  widely  advertised  and  utilized  for  simulation  research  are  the 
NRC  postdoctoral  fellowship  program  and  the  Intergovernmental  Personnel 
Act.  The  provisions  of  this  act  allow  for  exchange  of  senior  scien¬ 
tists,  which  is  crucial.  Reserve  Officers'  Training  Corps  (ROTC)  pro¬ 
grams  might  be  broadened  to  attract  science  graduates  for  military  re¬ 
search  on  simulation. 

The  comprehensive  research  program  needed  for  simulation  issues  to 
be  resolved  would  obviously  be  expensive  and  would  have  to  be  consid¬ 
ered  in  terms  of  potential  costs  and  benefits.  Simulator  training  af¬ 
fects  both  operational  readiness  and  pilot  safety.  Current  investment 
in  research  on  vision  and  training  issues  is  apparently  small  in  com¬ 
parison  to  the  projected  size  of  simulator  procurement  programs  (sev¬ 
eral  billion  dollars). 


SOME  STRATEGIES  FOR  RESEARCH 
Introduction 

Appendixes  A  through  F  provide  examples  of  research  areas  and  ap¬ 
proaches  that  might  help  elucidate  several  long-range  issues  confronted 
in  visual  simulation.  Primary  emphasis  is  given  to  low-level  flight, 
in  which  extraction  of  visual  information  from  terrain  features  is  cru¬ 
cial  but  little  understood.  Several  strategies  are  suggested  for  ex¬ 
ploring  which  visual  features  should  be  used  in  low-level  flight:  sys¬ 
tematic  condensation  of  opinions,  particularly  those  of  pilots  (appendix 
A) ,  geometric  analysis  of  terrain  information  potentially  usable  (ap¬ 
pendix  B) ,  and  psychophysical  analysis  of  visual  processing  modalities 
(appendixes  C  and  D) .  Appendix  E  examines  equipment  requirements  for 
display  of  whatever  visual  information  is  chosen.  It  also  addresses 
characteristics  of  visual  displays  that  limit  the  kind  of  information 
that  can  be  displayed.  Thus,  appendix  E  complements  the  other  appendixes 
and  is  applicable  to  simulation  of  any  flight  environment.  Several  of 
these  authors  have  suggested  particular  lines  of  research  that  could  be 
followed  within  their  paradigms. 

The  next  section  of  this  report  summarizes  the  theoretical  per¬ 
spective  and  working  assumptions  of  these  strategies  to  illustrate 
their  power  and  limitations.  An  extended  example  is  given  to  show  how 
these  strategies  might  be  combined  to  analyze  visual  information  re¬ 
quirements  for  low-level  flight.  The  complementarity  of  these  research 
strategies  is  emphasized. 


Identifying  Potentially  Critical  Factors  by 
Pooling  "Expert  Opinions" 

The  subjective  opinions  of  design  engineers,  pilots,  and  visual 
scientists  have  played  a  large  role  in  selecting  the  visual  features 
displayed  in  flight  simulators.  Many  attempts  to  identify  the  visual 
factors  or  cues  that  are  important  for  a  given  flight  task  have  begun 
with  a  subjectively  composed  list  of  scene  parameters  (e.g.,  field  of 
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view,  spatial  resolution,  luminance,  color,  representation  of  landscape, 
horizon,  clouds,  shading,  known  terrain  features,  shadows,  aerial  per¬ 
spective)  and  perceptual  factors  (e.g.,  binocular  vision,  accommoda¬ 
tion,  motion,  shear)  (Brown,  1976;  Gibson,  1950;  Kraft  et  al . ,  1980). 

The  many  attributes  and  factors  on  these  lists  may  be  subjected  to  ex¬ 
perimental  study,  but  often  they  are  simply  rank  ordered  subjectively 
and  incorporated  into  equipment  design  without  empirical  evaluation. 

This  procedure  has  several  flaws.  Some  pilots  may  have  useful  in¬ 
sight  about  the  visual  cues  they  use  in  given  flight  maneuvers  (e.g., 
Langewiesche ,  1944),  but  this  insight  is  generally  far  from  adequate 
for  characterizing  vision  information  requirements  for  display.  Fur¬ 
thermore,  pilots'  impressions  of  the  information  they  use  are  sometimes 
wrong  (see,  for  example,  Waag’s  (1981)  discussion  of  pilots'  evaluations 
of  platform  motion).  Casual  ranking  is  not  a  powerful  method  of  analyz¬ 
ing  opinions,  and  it  is  difficult  to  combine  systematically  the  opinions 
of  a  large  number  of  raters  in  this  way. 

Although  pilots'  opinions  are  inadequate  for  determining  vieual 
display  requirements,  they  may  be  highly  useful  in  identifying  features 
that  can  be  studied  experimentally.  More  effective  analysis  of  pilot 
opinions  could  be  achieved  with  non-metric  scaling  techniques.  One 
example  of  these  techniques,  KYST,  is  outlined  in  appendix  A.  To  sum¬ 
marize  KYST,  a  jury  of  knowledgeable  persons  is  asked  to  create  a  list 
of  all  the  scene  attributes  they  believe  apply  to  a  particular  flight 
task,  in  this  case  low-level  flight.  Pairs  of  these  attributes  are 
then  evaluated  as  to  their  importance  for  this  flying  task.  From  these 
ankings,  a  multidimensional  space  is  created  in  which  all  the  attri¬ 
butes  are  located  in  terms  of  their  importance  relative  to  one  another. 
This  space  can  then  be  reduced  by  factor  analysis,  with  the  result  be¬ 
ing  a  measure  of  how  many  dimensions  are  needed  to  capture  the  factors 
that  underlie  all  the  attributes  (Hake  and  Rodivan,  1966;  Kruskal  and 
Shepard,  1974).  Although  these  common  factors  may  not  be  identified, 
the  analysis  is  important  because  it  will  show  which  factors  might  be 
rejected  and  which  need  to  be  considered  seriously,  qualitatively  indi¬ 
cating  their  common  characteristics. 


Working  Assumptions 

If  a  visual  factor  is  missing  from  the  original  list  subjected  to 
KYST  analysis,  it  will  of  course  not  appear  in  the  final  analysis. 

This  absence,  however,  does  not  impede  relative  ranking  of  the  factors 
that  are  listed;  furthermore,  additional  factors  can  be  added  for  re¬ 
iterative  analysis.  Multidimensional  scaling  techniques  do  not  require 
that  the  list  of  "cues"  or  factors  be  exhaustive.  So  long  as  at  least 
one  item  on  the  list  of  attributes  captures  a  dimension  of  relevance, 
then  that  dimension  will  be  identified.  The  major  drawback  of  this 
technique  is  that  the  pair-wise  ranking  procedure  still  depends  on 
subjective  evaluations.  Pilots  may  be  unaware  of  their  use  of  some 
factors  and  may  make  unwarranted  assumptions  about  the  importance  of 
others.  Nevertheless,  pilots  undoubtedly  have  much  relevant  expertise 
that  is  hard  to  draw  on  adequately  without  some  sort  of  systematic 
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analysis  such  as  KYST.  By  suggesting  areas  for  experimental  study, 
this  kind  of  analysis  complements  other  research  approaches. 


Information  Analysis 


The  information  analysis  approach  attempts  to  define  the  visual 
information  requirements  needed  to  perform  a  flight  task.  For  example, 
during  low-level  flight,  the  pilot  must  judge  altitude  accurately  while 
flying  over  arbitrary  terrains.  Is  it  computationally  feasible  to  use 
stereopsis  to  solve  this  problem?  If  not,  which  sources  of  information 
could,  at  least  in  principle,  provide  sufficient  data  for  reliable  low- 
level  flight?  Is  it  practical  to  present  this  information  in  a  CIG 
display?  Appendix  B  gives  an  example  of  this  approach,  showing  that 
the  pilot  must  extract  three  types  of  information  about  the  terrain 
from  the  display:  shape,  scale,  and  orientation.  The  analysis  con¬ 
tinues  by  illustrating  which  simple  features  in  a  CIG  display  could 
provide  this  information  and  the  assumptions  the  pilot  (unconsciously) 
must  make  when  using  these  two-dimensional  image  features  to  make  three 
dimensional  (3-D)  inferences. 


Working  Assumptions 

Information  analysis  assumes  that  the  particular  forms  of  3-D  in¬ 
formation  that  the  pilot  extracts  from  the  CIG  display  may  be  individu¬ 
ally  isolated  along  with  the  visual  processes  that  extract  the  informa¬ 
tion.  It  is  further  assumed  that  those  processes  place  constraints  on 
the  interpretation  of  the  display,  and  that  these  constraints  may  be 
discovered.  These  constraints  are  provided  by  certain  geometrical  prop 
erties  of  surfaces  and  perspective  projection.  Human  vision  must  make 
assumptions  about  the  real  world  in  order  to  interpret  the  ambiguous 
visual  information  that  it  receives.  When  such  assumptions  are  wrong 
in  some  situations,  the  perception  does  not  correspond  to  reality. 

The  flight  simulator  generates  3-D  images  from  an  internal  model 
of  a  3-D  environment,  and  the  pilot's  visual  system  processes  these 
images  with  implicit  assumptions  about  the  nature  of  that  environment. 
If  the  pilot’s  3-D  perception  of  the  scene  is  to  coincide  with  that 
which  was  intended,  then  it  is  necessary  that  the  geometric  structure 
of  the  model,  and  the  way  it  is  portrayed  by  CIG,  obey  these  perceptual 
assumptions.  To  give  a  simplistic  but  illustrative  example,  we  often 
assume  that  intersecting  straight  lines  meet  at  right  angles  in  3-D, 
provided  there  is  no  evidence  to  the  contrary.  That  assumption  helps 
constrain  the  3-D  interpretation  of  intersections  in  an  image.  But 
if  the  internal  model  of  a  corner  of  a  field  or  runway  is  other  than  a 
right  angle  (imagine  that  the  runway  is  a  trapezoid),  then  the  viewer 
will  be  misled.  We  usually  have  several  independent  ways  of  determin¬ 
ing  the  3-D  shape,  hence  misinterpretation  is  not  a  serious  problem  in 
natural  scenes,  with  all  their  richness  and  redundancy.  But  in  the  im¬ 
poverished  scenes  of  a  simulator,  it  is  important  to  have  the  model's 
geometry  coincide  as  much  as  possible  with  that  expected  by  the  visual 
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system.  Consequently,  we  must  understand  the  computational  constraints 
(geometric  and  otherwise)  that  underlie  the  solution  to  a  vision  prob¬ 
lem  in  order  to  formulate  precisely  design  rules  that  govern  scene  gen¬ 
eration  and  lead  to  consistent  and  accurate  3-D  interpretations. 


Visual  Psychophysics 
Short-Term  Experimentation 

Multifaceted  experimental  analyses  (e.g.,  Westra  et  al.,  1981)  are 
useful  for  evaluating  the  relative  effects  of  identified  variables  on 
performance  in  simulators  or  on  transfer  of  training.  These  studies 
are  particularly  helpful  in  assessing  the  importance  of  the  differences 
between  subjects  relative  to  the  simulator  variables,  but  they  usually 
offer  little  insight  into  the  origin  of  the  individual  differences.  If 
different  pilots  rely  heavily  on  different  informational  aspects  of  the 
display,  then  this  knowledge  may  be  lost  in  a  factor  analysis.  To  un¬ 
derstand  the  degree  to  which  each  source  of  flight  information  can  be 
utilized  by  a  pilot,  long-term  psychophysical  studies  must  be  conducted. 


Long-Term  Experimentation 

Another  distinct  methodology  examines  certain  perceptual  abilities 
in  detail.  The  goal  is  to  derive  a  quantitative  understanding  of  the 
behavior  and  capabilities  of  the  human  visual  system.  Although  much  of 
psychophysics  has  dealt  with  the  resolving  power  of  the  visual  system 
(spatial  and  temporal),  much  recent  effort  has  concentrated  on  under¬ 
standing  particular  "modules"  of  the  visual  process,  such  as  stereopsis, 
texture,  motion,  and  looming.  By  understanding  the  capabilities  of 
various  "modules,"  their  utility  for  a  particular  flight  task  can  be 
evaluated . 

Studies  in  color  vision  provide  a  good  example  of  the  value  of  the 
modular  approach.  More  than  a  century  ago,  psychophysical  experiments 
demonstrated  that  the  human  color  vision  system  behaved  as  if  it  ana¬ 
lyzed  spectral  information  using  only  three  color  filters  or  "channels." 
This  property  of  the  eye  subsequently  was  exploited  in  color  photography 
and  makes  possible  practical  color  television  (as  well  as  CIG  color  dis¬ 
plays)  .  By  coding  the  signal  characteristics  to  match  those  of  the 
"channels”  of  the  human  color  system,  all  the  useful  information  about 
the  spectral  content  of  the  scene  is  delivered  economically  and  effec¬ 
tively  with  a  tremendous  economy  in  bandwidth.  More  recent  work  in 
vision  suggests  hints  of  other  such  modules  (Richards,  1980),  some  of 
which  may  be  especially  important  for  flying  tasks.  Appendixes  C  ar d 
D  illustrate  this  approach,  emphasizing  how  simple  tests  may  be  devel¬ 
oped  to  isolate  and  dissect  a  particular  module  and  to  assess  individual 
differences.  The  example  in  appendix  C  compares  sensitivities  of 
motion-in-depth  and  sizp  changes,  both  of  which  are  potent  sources  of 
information  useful  to  low-level  flight. 


Working  Assumptions 


The  crucial  assumption  that  distinguishes  a  multifactor  approach 
from  a  modular  one  is  that  the  modular  approach  assumes  that  the  visual 
ability  under  study  is  mediated  by  a  substantially  independent  visual 
process.  In  psychophysical  terms,  a  module  (or  set  of  channels)  pro¬ 
cesses  information  independently  of  another  set  of  channels  (i.e., 
movement  or  size  change) .  This  assumption  is  imposed  by  methodological 
pragmatics — there  are  analytic  means  for  studying  simple  processes  of 
few  parameters.  If  there  were  dependence  on  other  visual  processes, 
the  analysis  would  become  intractable. 

A  second  assumption  common  to  both  psychophysical  approaches  is 
that  the  visual  ability  under  study  is  relevant  to  the  task  of  flying 
an  aircraft.  This  assumption  is  made  initially,  when  first  studying 
the  ability,  but  later  that  assumption  may  or  may  not  be  verified.  The 
quantitative  understanding  gained  through  psychophysics  may  in  fact 
demonstrate  that  the  ability  is  irrelevant  to  the  task  of  flying.  On 
the  other  hand,  because  of  perceptual  sensitivity,  the  ability  may  pre¬ 
dict  special  relevance,  which  may  then  be  verified  in  a  transfer-of- 
training  study. 

The  third  assumption,  the  requirement  for  approximate  functional 
independence  of  different  modules  (at  least  at  the  early  levels  of  in¬ 
formation  processing) ,  places  limitations  on  the  range  of  visual  abili¬ 
ties  that  are  amenable  to  complete  psychophysical  study  using  a  modular 
approach. .  At  some  stage  of  visual  processing,  the  interactions  among 
the  individual  processes  become  significant,  and  at  that  point  any  modu¬ 
lar  analysis  by  psychophysics  becomes  infeasible. 


Integrating  the  Three  Approaches:  An  Example 
Minimal  Information 

One  approach  to  designing  a  simulator  is  to  analyze  the  minimal 
information  needed  to  perform  the  real-flight  task. 8  For  example,  if 
it  is  necessary  to  judge  altitude  at  some  point  during  visual  flight, 
there  must  be  adequate  information  about  altitude  in  the  visual  scene 
Although  the  minimal  information  required  for  simple  flying  maneuvers 


This  example  is  concerned  with  flight  performance  in  the  simulator. 

As  previously  discussed,  we  do  not  know  how  closely  performance  in  the 
simulator  must  follow  real-flight  demands  for  low-level  flight  to  ac¬ 
complish  transfer  of  training.  The  minimum  information  approach  is 
conservative  in  the  sense  that  it  should  provide  an  adequate  visual 
display  for  learning  skills  that  can  be  transferred  to  real  flight. 

We  do  not  preclude  the  possibility  of  demonstrating  transfer  of  train¬ 
ing  with  very  simple  visual  displays,  even  with  low-level  flight  (see 
previous  discussion  of  learning  of  perceptual  and  control  skills) . 

Such  empirical  demonstrations,  however,  may  be  difficult  to  generalize 
from  one  simulation  to  another. 
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such  as  horizontal  flight  or  landing  may  be  obvious,  one  of  the  diffi¬ 
culties  with  simulating  low-level  flight  is  that  this  minimal  scene 
content  is  not  known.  Here  intuition  is  not  an  adequate  guide,  for 
even  experienced  pilots  have  difficulty  in  isolating  and  formalizing 
what  is  needed  in  the  display  (Hennessy  et  al.,  1980).  A  non-metric 
multidimensional  scaling  analysis  could  be  a  useful  first  step  in  iden¬ 
tifying  relevant  attributes  of  the  environment  encountered  during  low- 
level  flight.  The  outcome  of  such  an  analysis  of  expert  opinion  should 
suggest  the  kind  of  information  required  by  the  pilot  to  fly  at  low 
altitudes.  For  example,  if  an  adequate  representation  of  the  ground 
plane  and  horizon  is  suggested  as  belonging  to  one  important  dimension 
of  the  analysis,  then  presumably  the  pilot  needs  to  know  his  orienta¬ 
tion  with  respect  to  the  ground  plane  for  successful  low-level  flight. 
If  "known  human  object  size"  or  "shadow"  appears  on  another  dimension, 
then  this  result  might  suggest  that  it  is  necessary  to  provide  some 
calibration  of  the  scale  of  the  terrain  so  that  altitude  can  be  judged 
reliably. 

We  will  assume  for  the  purposes  of  this  illustration  that  prelimi¬ 
nary  analysis  has  suggested  that  the  visual  scene  in  low-level  flight 
provides  at  least  three  kinds  of  information:  (1)  impact  point  and 
time,  (2)  altitude,  and  (3)  depression  of  flight  angle  with  respect  to 
the  ground. 


Informational  Analysis 

The  central  problem  of  low-level  flight  is  to  maintain  some  speci¬ 
fied  height  above  the  ground  (or  to  fly  within  an  altitude  band)  .  More 
specifically,  the  task  is  to  estimate  the  altitude  above  the  ground.  A, 
and  flight  path  angle  6  of  the  aircraft^  at  some  future  time  £  and  dis¬ 
tance  D,  where  the  time  _t  is  sufficiently  long  to  allow  a  corrective 
maneuver  to  be  made.  Figure  1  depicts  this  simple  situation,  in  which 
the  aircraft  must  fly  within  an  altitude  band  E_  over  hilly  terrain.  If 
the  pilot  has  complete  knowledge  of  the  performance  of  the  aircraft 
(e.g.,  its  climb  or  descent  rate),  one  can  specify  the  minimally  curved 
flight  contour  within  the  band  IS  (dotted  line) . 

The  problem  as  outlined  above  is  still  formidable  and  must  be 
broken  down  into  simpler  components.  Reviewing  the  goals  of  low-level 
flight,  it  is  clear  that  the  minimal  objective  is  to  avoid  hitting  the 
ground.  Perhaps  the  next  and  more  difficult  step  is  to  specify  the 
visual  information  needed  to  clear  the  peaks,  without  regard  to  accu¬ 
rate  altitude.  Finally,  the  information  requirements  for  a  specified 
altitude  objective  for  clearance  can  be  sought,  and  flight  within  an 
altitude  band  can  be  considered. 

A  breakdown  of  the  problem  suggests  the  following  states  of 
analysis : 


9 

Note  that  the  flight  path  is  not  generally  the  same  as  the  pitch  (or 
angle  of  attack)  of  the  aircraft. 


=  flight  path  angle ;  4>  ~  pitch  angle  (attitude).  Cross  hatching 
tes  desired  flight band;  heavy  dashed  line  is  minimal  flight  path. 
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1.  Derive  the  point  and  time  of  impact  for  the  current  flight 
t  rajectorv . 

2.  Specify  the  minimum  information  needed  to  calculate  depression 
angle  of  flight. 

3.  Specify  the  information  needed  to  clear  peaks. 

A.  Specify  the  information  needed  to  clear  peaks  at  200  feet 
+/-  100  feet . 

5.  Analyze  minimal  conditions  needed  to  fly  over  special  terrains 
within  an  altitude  band. 

6.  Extend  item  5  to  arbitrary  terrains  and  flight  conditions. 

We  will  now  use  the  derivation  of  the  impact  point  and  time  as  an 
example  of  an  information  analysis. 

Deriving  Point  and  Time  of  Impact  Deriving  the  point  and  time  of 

impact  is  relatively  straightforward,  provided  that  a  velocity  field 

of  sufficient  spatial  and  temporal  resolution  is  present.  (This  is  the 

"looming"  cue  investigated  by  Regan  and  Beverley,  1980,  and  Regan  et  al . , 

1981.)  The  impact  point  is  simply  the  source  of  the  flow  field,  as 

pointed  out  by  Gibson  (1950)  and  analyzed  by  Carel  (1961). 10  The  time 

of  impact  t-.  „  is  given  by 

r  —  impact  °  J 


t- .  =  r/0 

—  impact 


(1) 


where  0  is  the  radial  flow  rate  at  eccentricity  f  as  seen  by  the 
observer .  ^ 

To  solve  equation  1  and  to  determine  the  source  of  the  velocity 
field,  texture  and  contour  information  must  be  displayed  at  densities 
and  rates  compatible  with  the  performance  of  the  aircraft  and  skills  of 
the  pilot.  To  explore  these  visual  requirements,  we  can  examine  the 
information  content  needed  to  determine  impact  time. 

Figure  2  plots  the  flow  rate  versus  retinal  eccentricity  for  vari¬ 
ous  impact  times.  A  log  scale  has  been  used  to  make  the  curves  straight 
lines.  For  example,  consider  a  time  to  impact  of  500  milliseconds  (ms), 
which  might  roughly  represent  the  pilot's  reaction  time.  Figure  2  shows 
that  for  t^  =  500  ms,  flow  rate  changes  from  1/A  to  20  degrees/second 


Llewellyn  (1971)  and  appendix  C  report  evidence  that  human  observers 
cannot  identify  the  source  of  the  optic  flow  field.  However,  as  Harker 
and  Jones  (1980)  point  out,  in  flying  the  task  is  not  to  identify  the 
point  (center  of  expansion)  that  does  not  move,  but  rather  to  select  a 
point  and  keep  it  from  moving.  The  observer  conceivably  may  use  other 
sources  of  information  to  stabilize  the  expansion  point  and  then  ex¬ 
tract  flow  rate  information. 

^For  simplicity  of  illustration,  the  special  case  of  approach  to  an 
object  lying  in  the  observer's  frontal  plane  is  used.  For  the  more 
general  case  of  approach  to  a  slanted  plane,  equation  1  would  have  to 
include  terms  for  the  effects  of  foreshortening. 
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FIGURE  2  Flow  rate  versus  retinal  eccentricity  for 
several  impact  times. 
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(deg/s)  in  the  central  1/8  to  10  deg  (this  is  within  the  range  of  human 
visual  sensitivity  to  motion).  If  the  display  is  a  1-meter  (m)  diameter 
CIG  with  a  30-ms  update  rate  and  a  1,000-line  interlaced  raster  at  1  m, 
then  the  fastest  motion  that  can  be  generated  without  skipping  a  raster 
line  is  6  deg/s.  Thus,  to  utilize  a  500-ms  reaction  time  capability, 
texture  contours  would  have  to  occur  within  3  deg  eccentricity.  One 
might  want  to  provide  perhaps  ten  times  this  contour  density  (i.e., 

3  cycles/deg)  in  the  visual  display  to  allow  effective  estimation  of 
flow  rate  and  angle.  Without  this  display  capability,  the  simulator 
will  not  provide  all  the  useful  information  offered  by  equation  1. 

(See  appendix  E  for  other  design  considerations.)  (As  an  aside,  we 
note  that  this  stringent  display  requirement  is  needed  only  at  the  cen¬ 
ter  of  the  display.  Since  the  peripheral  display  requirement  is  much 
less  severe,  peripheral  edges  might  be  traded  for  central  ones,  in  some 
graded  manner  according  to  figure  2  and  human  abilities.)  This  alloca¬ 
tion  of  "center"  of  course  would  depend  on  where  the  pilot  looks. 

The  above  is  an  example  of  how  an  information-theoretic  analysis 
aimed  at  designing  a  display  to  help  the  pilot  avoid  impact  should  pro¬ 
ceed.  If  sufficient  information  about  the  time  of  impact  is  not  dis¬ 
played,  the  pilot  might  adopt  a  conservative  strategy,  such  as  always 
keeping  the  flight  path  above  the  terrain,  but  this  strategy  precludes 
controlled  terrain  following.  Is  this  conservative  behavior,  elicited 
by  the  inadequacies  of  the  simulator,  the  best  training  for  the  pilot? 

If  it  is  not,  then  the  capabilities  of  the  simulator  should  be  upgraded 
to  provide  at  least  the  minimum  information  content  needed  to  compute 
impact  point  and  time. 

12 

Information  Needed  to  Clear  Peaks  To  hint  at  possible  solutions 
to  the  increasingly  difficult  stages  of  the  problem,  we  briefly  con¬ 
sider  some  additional  information  for  clearing  peaks. 

Note  that  if  the  sky  is  textureless,  then  the  source  of  the  flow 
field  must  be  inferred  from  the  flow  pattern  of  the  terrain  contours. 
Once  again,  therefore,  an  analysis  of  the  display  requirements  should 
be  made  to  ensure  that  the  terrain  content  does  indeed  provide  suffi¬ 
cient  spatial  resolution  and  detail  to  allow  such  an  extrapolation  to 
be  made . 

Without  such  information,  a  pilot  is  forced  to  look  for  other  cues 
as  to  direction  of  flight.  One  possibility  is  to  direct  the  aircraft 
so  that  the  peak  of  the  nearest  hill  is  aligned  with  the  next  most  dis¬ 
tant  peak. 13  However,  without  an  adequate  velocity  field,  neither  the 
clearance  altitude  nor  the  time  of  arrival  at  the  peak  can  be  recovered. 
Again,  an  information-based  analysis  similar  to  that  described  earlier 

__ 

For  suggestions  regarding  the  extraction  of  altitude,  see  Marker  and 
Jones  (1980) . 

13 

Note  that  the  alignment  of  the  aircraft  with  a  near  ridge  and  the  next 
peak  is  a  good  strategy  for  ensuring  that  the  flight  path  angle  is  nega¬ 
tive  with  respect  to  the  ground  plane. 
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can  specify  the  minimal  display  conditions,  given  the  aircraft  perfor¬ 
mance  and  the  reaction  times  and  visual  abilities  of  the  pilot. 

Terrain  Following  Finally,  if  the  full  mission  is  to  flv  in  the 
band  II  of  figure  1  as  originally  proposed,  then  clearly  the  pilot  must 
be  able  to  perceive  terrain  shape  fairly  accurately.  At  present,  it 
appears  that  the  information  required  to  do  this  is  beyond  reasonable 
technical  capability,  given  the  demands  of  low-level  flight  and  the 
present  optical  and  computational  limitations  of  the  available  graphics 
displays.  However,  a  computational  analysis  could  profitably  explore 
the  possibility  of  a  sufficiently  rich  display  region  for  the  lower 
portion  of  the  display.  Such  a  region  might  include  only  a  small  frac¬ 
tion  of  the  total  display  field,  allowing  the  possibility  of  at  least 
an  order  of  magnitude  increase  in  texture  content  in  a  critical  visual 
sector  with  little  loss  elsewhere.  (See  appendix  D  for  other  field-of- 
view  considerations.) 

Summary  of  the  Approach  The  design  of  an  optimally  efficient  simu¬ 
lator  display  requires  an  information  analysis  of  the  task  that  keeps 
in  mind  the  performance  of  the  aircraft,  the  visual  abilities  of  the 
pilot,  and  the  display  capabilities.  The  formal  steps  in  such  an  analv- 
sis  are  as  follows: 

1.  A  clear  statement  of  the  ultimate  objective  (task). 

2.  The  subdivision  of  this  objective  into  the  component  problems, 
from  the  simplest  to  the  most  complex. 

3.  An  analysis  of  the  minimal  information  requirements  needed  to 
solve  the  simplest  problem. 

4.  An  extension  of  this  analysis  to  the  more  complex  cases,  shov¬ 
ing  how  the  required  variables  can  be  extracted  from  the  dis¬ 
played  scene. 

5.  Consideration  of  the  display  and  human  capabilities  that  limit 
the  available  ranges  of  the  variables  of  interest. 


Applying  Psychophysics 

The  non-metric  scaling  technique  is  designed  to  suggest  which  as¬ 
pects  of  the  environment  are  important  to  a  task  such  as  low-level 
flight.  The  information  analysis  then  shows  what  must  be  included  in 
a  two-dimensional  visual  display  of  the  simulator  to  allow  a  pilot  to 
extract  this  information.  However,  because  the  results  of  each  approach 
depend  on  certain  assumptions,  it  is  necessary  to  demonstrate  that  the 
observer  does  indeed  have  the  ability  to  sense  this  information  with 
sufficient  accuracy.  For  example,  a  high-resolution  binocular  visual 
system  can,  in  principle,  recover  a  complete  3-D  terrain  map  and  hence 
reveal  distances,  slopes,  altitudes,  and  velocities.  The  threshold  for 
human  stereoacuitv,  however,  limits  the  range  of  binocular  vision  to 
distances  that  would  be  covered  by  a  high-speed  aircraft  in  seconds. 

A  critical  component  of  the  analysis  of  a  problem  to  be  simulated  is 
to  ensure  that  the  displayed  information  is  not  only  relevant  but  use¬ 
ful.  At  the  same  time,  psychophysical  studies  of  the  parameters  of 
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interest  have  the  potential  of  foretelling  whether  one  individual  will 
perform  better  than  another  on  a  well-defined  and  understood  task 
(Regan  et  al.,  1981).  We  will  illustrate  these  points  using  the  sim¬ 
plified  low-level  task  of  deriving  the  time  for  impact.  (For  addi¬ 
tional  experimental  suggestions  regarding  the  position  of  impact  (or 
flight  direction),  see  appendix  D.) 

Time  to  Impact  Time-to-impact  information  is  contained  in  the 
optical  flow  field,  but  can  the  human  observer  use  this  information? 

If  so,  is  all  the  information  processing  capability  of  the  observer 
utilized  in  the  simulator?  A  simple  experiment  by  Carel  (1961)  sug¬ 
gested  that  time  to  impact  can  indeed  be  estimated  as  predicted  from 
equation  1,  without  knowledge  of  the  distance  to  the  surface,  object 
size,  or  speed.  In  this  laboratory  study,  with  an  expanding  flow  field 
of  random  dots,  subjects  could  easily  make  extrapolations  to  the  time 
to  impact  when  the  field  expansion  was  stopped  before  simulated  impact 
(see  figure  3).l^  The  range  of  judgments  is  substantial,  however.  Is 
this  range  due  to  a  limitation  of  the  apparatus  or  laboratory  setting, 
or  is  it  an  inherent  inability  of  the  human  visual  system?  To  answer 
this  question  requires  further  psychophysical  study  that  fits  nicely 
into  the  module  or  "channel"  approach. 

Size-Change  Channels  Referring  again  to  figure  2,  we  note  that 
the  critical  information  about  time  to  impact  is  conveyed  by  a  change 
in  angular  size  at  a  given  eccentricity.  To  solve  equation  1,  we  must 
either  be  able  to  measure  both  the  rate  of  change  in  angular  size,  9, 
and  the  retinal  eccentricity,  6,  and  then  take  the  ratio  or  measure 

directly.  For  illustration,  we  will  assume  that  the  human  observer 
can  reliably  recover  e  and  examine  the  psychophysical  constraints  im¬ 
posed  on  displaying  and  using  the  derivative  9,  recognizing  that  the 
more  useful  parameter-^  might  be  9/9. 

Human  capability  of  measuring  size  change  (6)  has  been  studied  by 
Regan  and  Beverley  (1979).  Figure  4  shows  threshold  response  data  that 
Regan  and  Beverley  discuss  in  terms  of  tuning  characteristics  of  a 
size-change  channel.  Optimal  sensitivity  for  all  three  observers  is 
near  a  rate  of  3  Hz  for  1/4  min  arc,  corresponding  to  a  velocity  of 
1/80  deg/s.  (A  more  conservative  threshold  rate  estimate  for  most  of 


This  does  not  imply  that  the  observer  necessarily  could  identify  the 
point  of  impact  from  the  flow  field  (see  footnote  10).  Furthermore, 
ambiguous  and  erroneous  perception  of  expanding  patterns  is  possible, 
as  illustrated  by  the  Ames  trapezoid  illusion.  Even  with  full  optical 
flow  patterns,  what  is  perceived  may  depend  on  the  state  of  the  observ¬ 
er,  particularly  when  the  scene  content  is  impoverished  in  other  re¬ 
spects.  Further  experimentation  is  needed  to  relate  these  laboratory 
data  to  visual  simulation  (a.g.,  Harker  and  Jones,  1980). 

^To  test  whether  the  human  observer  uses  9  or  6/9,  one  can  compare  the 
combined  variance  for  estimating  9  and  6  separately  with  that  for  esti¬ 
mating  the  ratio  9/0. 
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FIGURE  3  Carel's  (1961)  data  for  the  reliability  of 
estimating  impact  time. 
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the  frequency  range  would  be  1/10  deg/s.)  If  the  full  capability  of 
the  human  observer  is  to  be  utilized  to  extract  impact  time  from  rate 
change,  then  both  the  raster  size  and  rate  must  meet  these  limits. 

For  a  450  raster-line  display  at  1  m,  the  raster  would  be  10  lines  per 
degree  per  meter  of  display.  This  translates  into  6  min  between  raster 
lines  for  aim  display,  so  l/25th  m  display  diameter  would  be  required 
if  the  full  capability  of  the  human  observer  is  to  be  used  to  solve 
for  time  of  impact. 

Similar  psychophysical  constraints  can  be  determined  to  set  an 
upper  bound  on  the  detectable  change  in  size,  using  the  readily  avail¬ 
able  spatial-temporal  contrast  sensitivity  functions  (Kelly,  1979; 

Wilson  and  Giese,  1977).  Since  these  functions  change  with  retinal 
eccentricity,  so  will  the  limits  for  the  useful  flow  rates. 

Figure  5  illustrates  how  human  capabilities  might  restrict  the 
range  of  flow  rates  that  are  useful  for  calculating  impact  time.  The 
minimum  spatial  resolution  for  the  human  observer  is  also  included  to 
show  how  the  available  flow  rates  are  restricted  to  a  portion  of  the  en¬ 
tire  range.  The  useful  region  is,  of  course,  based  on  normative  data. 
Regan  and  coworkers  (1981)  have  shown  that  on  related  tasks,  individual 
differences  may  run  as  high  as  80  to  1,  suggesting  a  wide  range  of 
abilities  for  successful  crash-avoidance  during  low-level  flight  (see 
appendix  C) .  These  large  individual  differences  may  well  be  responsi¬ 
ble  for  the  wide  range  in  Carel's  (1961)  data  previously  discussed 
(figure  3).  However,  further  work  is  needed  to  establish  the  role  of 
size-change  mechanisms  and  to  determine  whether  they  are  critical  for 
low-level  flight. 

Evaluating  "Optic  Flow"  The  expanding  pattern  created  by  forward 
locomotion  includes  a  plethora  of  information  about  the  world  and  one's 
motion  in  that  world.  The  problem  is  to  determine  how  the  retinal  pat¬ 
tern  can  be  decoded  to  recover  the  aspects  of  the  world  that  are  rele¬ 
vant  to  the  tasks  at  hand.  Although  some  progress  has  been  made  in 
this  area  (Longuet-Higgins  and  Prazdny,  1980;  Oilman,  1979),  still  more 
theoretical  and  psychophysical  work  is  needed.  The  brief  information 
analysis  in  the  preceding  section  suggests  that  the  calculation  of  the 
flight  angle  from  the  flow  pattern  would  be  the  next  problem  to  tackle 
after  the  derivation  of  impact  time.  To  solve  this  problem,  the  ground 
plane  must  be  estimated  from  the  retinal  flow  pattern.  Appendix  D  dis¬ 
cusses  some  aspects  of  this  problem.  A  study  by  Harker  and  Jones  (1980) 
is  also  relevant.  Rather  than  exploring  various  parameters  of  the 
retinal  flow  field  solely  with  psychophysics,  however,  we  suggest  com¬ 
bining  research  strategies,  as  illustrated  in  this  report. 


Evaluating  Vision  Research 

The  integrated  research  strategy  illustrated  above  should  provide 
a  powerful  approach  to  the  fundamental  problems  of  developing  more  ef¬ 
fective  visual  displavs  for  simulation.  This  strategy  suffers,  however, 
from  the  "divide  and  conquer"  approach  typically  used  in  the  laboratory 
to  isolate  and  study  individual  processes.  This  approach  fails  to 
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examine  visual  perception  in  its  natural  state,  in  which  isual  input 
is  usually  rich  and  redundant.  Vision  is  not  merely  a  collection  of 
independent  processes.  Sensory  information  from  the  eyes  is  integrated 
in  a  complex  manner  with  knowledge  of  the  world  and  expectations  of  what 
will  be  seen,  which  are  continuously  modified  as  the  observer  interacts 
with  the  environment.  Thus,  vision  is  intimately  tied  to  sensorimotor 
behavior  and  intellectual  problem  solving.  Studies  of  the  individual 
processes  that  derive  information  from  the  eyes  do  not  shed  light  on 
these  interactions.  The  human  ability  to  use  various  sources  of  infor¬ 
mation  and  a  range  of  strategies  to  perform  real-world  tasks  suggests 
that  more  than  one  approach  to  visual  simulation  of  a  given  flight  task 
may  be  effective.  It  is  not  possible  to  determine  the  most  effective 
approach  to  visual  simulation  solely  from  considerations  of  vision. 

All  of  this  indicates  that  suggestions  arising  from  this  or  any  other 
research  strategy  will  have  to  be  empirically  evaluated  to  determine 
choir  value  for  simulation  flight  training. 
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APPENDIX  A 

ANALYSIS  OF  NON-METRIC  DATA 
D.  H.  Fender 


It  seems  almost  a  truism  that  pilots  must  know  a  great  deal  about 
those  aspects  of  the  visual  environment  that  contribute  to  the  success¬ 
ful  performance  of  their  task.  Simulation  researchers,  however,  report 
that  the  subjective  assessments  of  pilots  are  not  a  reliable  guide  to 
the  visual  information  actually  used  in  flying. 

My  evaluation  of  the  problem  is  that  pilots  are  probably  not  con¬ 
sciously  aware  of  what  they  know  about  this  problem,  cannot  turn  this 
knowledge  into  reliable  verbal  forms,  and  that  the  information  is  prob¬ 
ably  non-metric  anyway.  I  persist  in  my  belief  that  there  must  be  a 
mine  of  information  residing  with  pilots;  the  major  problem  is  to  de¬ 
vise  reliable  ways  of  tapping  and  analyzing  this  information. 

There  are  two  problems  here — how  to  tap  the  information  and  how  to 
analyze  it.  Let  me  deal  with  the  second  problem  first.  Since  the  data 
are  most  likely  non-metric,  the  analytic  method  of  my  choice  would  be 
multidimensional  scaling  with  an  unknown  proximity  function,  as  sug¬ 
gested  by  Shepard  (1962)  .  This  analytic  method  requires  that  we  have  a 
confusion  matrix  (or  proximity  matrix)  between  the  nonparametric  prop¬ 
erties  of  the  simulator  visual  environment  that  we  wish  to  study,  and 
this  dictates  the  form  of  experiment  that  we  should  do  to  tap  the  pi¬ 
lot's  experience. 

For  example,  if  we  wish  to  analyze  the  present  simulator  we  might 
start  by  asking  the  subjects  to  write  down  a  list  of  all  the  attributes 
they  can  think  of  that  apply  to  the  visual  environment  of  the  simulator. 
We  would  probably  get  a  pretty  conventional  list,  such  as 

(a)  field  of  view 

(b)  monochromatic  display 

(c)  collimated  viewing  condition 

(d)  resolution  of  the  display 

(e)  representation  of  landscape 

(f)  movement  of  display  contingent  on  maneuvers  of  plane, 
and  a  few  unusual  items,  such  as 

(g)  fingerprints  on  cockpit  canopy 

(h)  flicker  in  peripheral  vision 

Some  might  even  abject  to  the  projective  geometry  used  in  the  display, 
and  record 

(i)  Euclidean  geometry! 
and  so  on. 
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We  would  then  take  the  combined  lists  of  all  of  the  subjects,  pos 
siblv  add  a  few  attributes  of  our  own,  and  then  put  the  following  prob 
lent  to  the  subjects: 


In  the  design  of  a  future  simulator  we  shall  not  be  able  to  in¬ 
corporate  all  the  attributes  of  the  one  with  which  you  a’-e 
familiar.  You  will  therefore  be  asked  to  select  between  pairs 
of  attributes.  Your  task  is  to  say  which  of  the  pair  you  would 
retain  in  the  new  design.  We  make  the  assumption  that  the  at¬ 
tribute  you  do  not  select  will  be  (i)  eliminated  (as  in  the 
case  of  platform  motion),  (ii)  degraded  so  as  to  be  useless  in 
the  performance  of  the  task  (as  in  the  case  of  resolution)  cr 
(iii)  changed  so  that  the  effect  on  you  becomes  objectionable 
(as  in  the  case  of  peripheral  flicker).  In  short,  a  vote 
against  an  attribute  is  always  to  be  interpreted  as  "change  the 
attribute  so  as  to  degrade  the  performance  of  the  flying  task." 
Thus  a  vote  against  fingerprints  does  not  mean  "remove  them," 
it  means  "make  them  worse."  Some  pairs  may  seem  to  represent 
an  impossible  choice,  for  example,  contrast  versus  resolution, 
since  complete  degradation  of  either  will  kill  the  visual  in¬ 
put;  but  please  make  a  choice  anyway! 


After  the  experiment  has  been  performed,  using  for  example  a  popu 
lation  of  twenty,  we  would  have  a  matrix  such  as  follows,  where  the 
number  in  each  cell  represents  the  number  of  subjects  voting  for  that 
condition: 
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A  consideration  of  these  numbers  alone  begins  to  give  some  idea  of 
how  the  subjects  perceive  the  flying  task;  for  example,  twenty  voted  to 
exclude  (c)  in  relation  to  (a),  but  equally  twenty  voted  to  include  (c) 
when  compared  with  (d) .  Already  we  begin  to  see  the  relative  merits  of 
(a),  (c) ,  and  (d)  insofar  as  the  subjects  are  concerned;  we  could  pos¬ 
sibly  lay  them  out  along  a  decision  axis,  such  as  that  shown  below: 


(a)  ♦ - 20 - *•  (c)  *-■ 

^ - 

Keep 


20 - *>(d) 

Decision 

— - - *  parameter 

Reject 


But  this  is  patently  wrong  when  we  consider  (d)  against  (a) — nineteen 
voted  to  keep  (d)  and  reject  (a)!  It  is  probable  that  the  (a,d)  de¬ 
cision  was  made  on  a  different  parameter  from  that  used  in  the  (a,c) 
decision  or  the  (c,d)  decision.  In  this  case  a  better  layout  might  be 
as  follows: 


Keep 


Decision 
parameter  2 

Reject 


Keep  Reject  parameter 


Obviously,  the  geometry  doesn't  quite  work  out,  but  it  might  be  possi¬ 
ble  to  juggle  the  layout  so  that  at  least  the  rank  ordering  of  the  in¬ 
terpoint  distances  is  monotonically  related  to  the  experimental  data. 

We  might  be  able  to  do  this  in  our  heads  for  a  small  number  of  data 
points,  but  the  complexity  increases  rapidly  with  the  number  of  points. 

For  a  group  of  N  points,  any  prescribed  rank  order  of  the  N(N-l)/2 
interpoint  distances  (including  ties)  is  realizable  in  an  Euclidean 
metric  space  provided  it  is  of  dimension  N-l  or  greater.  Suppose  that 
we  had  tested  nineteen  attributes;  this  means  we  could  display  the  171 
proximity  measures  as  metric  distances  among  nineteen  points  in  an  18- 
dimensional  space.  Apart  from  the  obvious  problem  of  visualizing  an 
18-dimensional  space,  such  a  configuration  would  yield  little  more  in¬ 
formation  than  the  simple  rank  ordering  of  the  171  proximities.  It  is 
therefore  highly  desirable  to  reduce  the  order  of  our  metric  space  if 
we  can  do  so  without  distorting  the  representation  of  our  proximity 
measures . 

The  mechanism  for  reduction  of  dimensionality  proposed  by  Kruskal 
(1964)  is  based  on  finding  a  structure  that  maintains  the  monotonicity 
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condition.  For  a  given  configuration  in  a  space  of  dimension  n,  where 
n  i.  N=l,  a  monotone  regression  of  distance  upon  proximity  measure  is 
performed  and  the  residual  variance  used  as  a  measure  of  how  well  the 
particular  configuration  satisfies  the  monotonicity  condition.  This 
measure  Kruskal  calls  the  stress.  The  configuration  is  changed  itera¬ 
tively  following  a  nonlinear  minimization  algorithm  based  on  the  method 
of  steepest  descent  until  the  configuration  with  the  smallest  possible 
stress  is  obtained  for  that  order  space.  Hence,  the  dimensionality  of 
the  space  may  be  reduced  so  long  as  a  configuration  can  be  found  that 
yields  a  value  of  stress  that  is  acceptably  small.  If  this  condition 
is  satisfied,  then  the  structure  in  the  lower-order  space  is  just  as 
valid  a  representation  of  the  proximities  as  the  18-dimensional  struc¬ 
ture.  A  large  stress  value  implies  monotonicity  is  not  maintained  and 
hence  the  configuration  does  not  very  well  represent  the  psychophysical 
data . 

It  is  of  considerable  interest  to  determine  the  minimum  dimen¬ 
sionality  space  in  which  a  configuration  can  be  found  that  adequately 
represents  the  psychological  proximities.  The  number  of  dimensions 
required  can  then  be  associated  with  the  number  of  fundamental  proper¬ 
ties  of  the  stimuli  upon  which  the  discriminations  art.  based.  But  it 
is  alsc  important  to  remember  that  this  scaling  technique  is  only  a 
concept  ;al  convenience;  visualization  of  a  configuration  ’n  a  low-order 
space  is  facilitated  and  can  reveal  important  aspects  of  the  interpoint 
relations . 

I  have  used  multidimensional  scaling  in  a  number  of  contexts,  and 
my  experience  has  been  that  the  human  can  make  decisions  only  in  a  space 
of  very  low  dimensionality.  Typically,  the  curves  of  stress  versus  di¬ 
mensionality  in  an  experiment  designed  to  differentiate  between  visual 
textures  (Santoro  and  Fender,  1976)  look  thus: 


•  =  subject  1.  0=  subject  2.  0=  subject  3. 


Curves  such  as  these  indicate  that  a  3-dimensional  configuration 
would  probably  describe  the  data  adequately,  and  a  model  of  higher  di¬ 
mensionality  would  overfit  the  data. 
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It  is  at  this  point  that  the  wit  of  the  experimenter  enters  the 
problem.  The  computer  merely  churns  out  a  mass  of  numbers;  the  nine¬ 
teen  attributes  are  positioned  in  n-space  so  as  to  minimize  the  stress, 
where  n  is  chosen  by  the  experimenter  from  a  curve  such  as  the  one 
above.  The  computer  gives  the  coordinates  of  the  points  representing 
the  attributes.  For  simplicity  of  visualization,  let  us  assume  that  we 
can  represent  the  data  in  3-space.  We  next  perform  a  varimax  rotation, 
that  is,  we  find  the  axis  such  that  if  the  data  points  were  projected 
onto  a  plane  perpendicular  to  this  axis,  the  variance  of  the  data  would 
be  reduced  by  the  maximum  amount.  The  two  other  orthogonal  axes  satis¬ 
fying  the  same  condition  are  also  calculated.  We  then  have  to  think. 
Suppose  that  the  data  were  as  follows  (reduced  to  2-space  for  further 
ease  of  visualization) : 


That  is,  the  data  points  lie  very  neatly  along  the  varimax  axes  AA  and 
BB.  We  obviously  have  two  intermingled  classes  of  data,  one  differen¬ 
tiated  by  whatever  property  is  mapped  along  AA  and  the  other  by  the 
property  mapped  along  BB.  The  experimenter  then  has  to  examine  the 
attributes  that  are  mapped  along  AA;  only  his  insight  will  help  him 
understand  what  property  lies  along  AA  and  is  scaled  such  that  the  at¬ 
tributes  at  one  end  (the  analysis  doesn't  tell  you  which  end!)  have  "a 
lot"  of  this  property  while  those  at  the  other  end  have  "not  very  much" 
of  the  same  property.  The  same  process  can  be  performed  along  BB.  The 
analytical  thought  process  becomes  more  complex  if  the  data  points  are 
as  shown : 


B 
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That  is,  there  are  many  points  such  as  C  that  do  not  lie  on  AA  or  BB. 

The  point  C  can  have  any  lot /little  combination  of  the  properties  AA  and 
BB,  since  we  do  not  know  a  priori  which  way  the  properties  are  scaled 
along  the  axes.  In  general,  one  uses  the  points  lying  on  axis  to  de¬ 
velop  ideas  about  the  property  mapped  along  each  axis,  and  then  one 
checks  the  off-axis  points  against  these  theories.  Of  course,  it  may 
happen  that  no  points  in  the  data  structure  lie  on  axis! 

The  outcome  of  this  analysis — which  is  not  exact  by  any  means — is 
a  strong  indication  of  the  parameters  the  subjects  used  when  making 
their  decisions  to  accept  or  reject  an  attribute.  Whether  these  de¬ 
cisions  have  any  link  with  flying  a  simulator  depends  on  the  skill  with 
which  the  original  experiment  was  set  up.  But  suppose  that  the  work  is 
well  done;  the  design  principle  is  then  self-evident — add  (or  enhance) 
features  whose  attributes  lie  at  the  "good"  end  of  the  longest  axis, 
and  if  compromises  have  to  be  made,  lop  off  features  whose  attributes 
lie  at  the  "bad"  end  of  any  axis. 

The  most  evident  criticism  of  this  approach  is  that  if  an  attri¬ 
bute  is  not  included  in  the  original  list,  it  will  not  appear  in  the 
final  analysis.  This  is  a  potent  criticism  but  fortunately  not  a  fatal 
one — the  analysis  proceeds  perfectly  well  without  the  missing  attribute, 
and  in  many  cases  an  examination  of  the  axes  of  the  final  structure 
prompts  the  experimenter  in  regard  to  attributes  that  may  have  been 
missed.  The  work  can  then  be  repeated  in  an  iterative  fashion. 

The  experiment  described  earlier  is  put  forward  only  as  an  example 
of  how  multidimensional  scaling  might  be  applied.  Many  other  experi¬ 
mental  arrangements  are  possible,  and  workers  more  closely  in  touch 
with  the  simulator  problem  are  surely  the  ones  to  design  the  actual  ex¬ 
periment.  For  example,  rather  than  giving  a  colloquial  explanation  of 
what  a  vote  against  a  particular  attribute  might  mean,  it  may  be  better 
to  seat  the  subject  in  the  simulator  and  say,  "If  you  vote  against 
resolution,  this  is  what  will  happen,"  then  tweak  the  knob  to  blur  the 
image,  and  so  on  over  all  the  attributes,  before  the  subject  is  asked 
to  cast  his  vote. 

Further,  instead  of  adjudicating  between  attributes  in  a  simula¬ 
tor,  the  subject  might  be  asked  to  evaluate  an  attribute  in  the  simu¬ 
lator  against  the  same  attribute  in  a  real  plane,  or  in  another  model 
of  simulator,  on  a  scale  of  one  to  five,  for  example. 

A  program  to  perform  this  analysis  is  available  commercial Iv .  It  has 
many  features  not  treated  here.  They  are  described  in  Kruskal  et  al . 
(1973).  One  feature,  however,  is  worth  mentioning.  The  program  has  the 
ability  to  analyze  data  represented  by  two  or  more  proximity  matrices 
simultaneously.  That  is,  it  derives  the  structure  that  is  the  best  fit 
to  both  matrices.  One  way  to  exploit  this  would  be  as  follows:  Tf  anv 
objective  data  exist  on  the  merits,  or  relative  merits,  of  anv  of  the 
attributes,  these  could  be  plugged  into  a  second  matrix.  It  is  not 
necessary  to  fill  all  the  cells  of  the  matrix.  The  program  works  quite 
well  on  an  incomplete  matrix.  IF  the  analvsis  is  then  run  the  two 
matrices,  using  varying  weighting  factors  between  them,  at  the  least,  it 
is  possible  to  test  the  concordance  of  the  subiective  and  of  the 
objective  data. 
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In  summary,  I  propose  that  the  method  of  multidimensional  scaling 
provides  a  technique  that  could  be  applied  rapidly  and  at  low  cost  to 
the  backlog  of  objective  information  concerning  flight  simulators  and 
to  subjective  information  that  could  be  gathered  without  too  much  ex¬ 
perimental  effort.  At  worst,  the  analysis  might  tell  us  no  more  than 
we  already  know,  but  at  best  it  might  give  considerable  insight  into 
the  functional  attributes  of  a  flight  simulator  and  of  the  interactions 
between  them. 

Other  aspects  of  multidimensional  scaling  are  explored  in  Shepard 
(1974),  Shepard  et  al .  (1972),  Kruskal  and  Shepard  (1974),  and  Egan 
(1971). 
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APPENDIX  B 

COMPUTATIONAL  ANALYSIS:  IMPLICATIONS  FOR 
VISUAL  SIMULATIONS  OF  TERRAIN1 

Kent  A.  Stevens 


Introduction 

Low-level  flight  (LLF)  is  a  flight  regime  of  increasing  importance 
in  combat  and  also  a  regime  where  significant  difficulty  has  been  ex¬ 
perienced  in  its  simulation.  Pilots  who  are  expert  in  actual  LLF  often 
cannot  duplicate  their  performance  in  the  simulator.  When  "on  the  deck 
in  the  simulator  they  have  difficulty  both  in  following  the  contour  of 
the  terrain  at  low  altitude  (without  crashing)  and  in  judging  altitude. 
In  part,  the  difficulty  is  technological;  for  instance,  the  very  rapid 
motion  of  terrain  detail  across  the  display  requires  fast  display  pro¬ 
cessors.  Other  difficulties  are  perceptual:  the  visual  displays  must 
convey  an  adequate  3-D  understanding  of  the  terrain  over  which  the  pi¬ 
lot  must  fly.  The  combination  of  technological  and  perceptual  problems 
has  prompted  some  reexamination  of  the  methods  for  terrain  depiction. 
This  paper  describes  a  new  technique  that  may  improve  the  visual  simu¬ 
lation  of  terrain,  as  applied  to  the  simulation  of  LLF. 

The  introduction  will  discuss  three  background  issues.  The  first 
concerns  flight  training.  Research  and  development  efforts  regarding 
simulators  for  flight  training  should  be  evaluated  in  terms  of  their 
impact  on  training.  However,  LLF  simulation  presents  some  basic  visual 
problems  that  must  be  solved  regardless  of  whether  the  simulator  is 
used  for  training  or  any  other  purpose.  Hence  we  concentrate  here 
solely  on  the  visual  problems  of  LLF  simulation.  Next,  we  show  that 


this  is  a  revision  of  the  paper  that  appeared  in  the  Image  Generation/ 
Display  Conference  II  in  June  1981.  A  number  of  people  made  important 
contributions  and  suggestions:  Prof.  W.  Richards  and  Drs .  S.  Collyer, 
D.  Regan,  J.  Richter,  and  S.  Ullman.  The  author  gratefully  acknowl¬ 
edges  the  useful  comments  on  an  earlier  draft  of  this  article  provided 
by  Drs.  K.  Dismukes,  R.  Haber,  and  J.  Hochberg.  The  manuscript  was  pre 
pared  at  the  Artificial  Intelligence  Laboratory  of  the  Massachusetts 
Institute  of  Technology,  with  support  provided  in  part  by  the  Advanced 
Research  Projects  Agency  of  the  Department  of  Defense  under  Otf'ce  of 
Naval  Research  contract  N00014-75-C-0643  and  in  part  by  the  Air  Force 
Office  of  Scientific  Research  and  the  National  Science  Foundation  under 
MCS79-2  3110 . 
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although  real  world  scenes  are  distorted  in  computer  image  generation 
(CIG)  displays,  they  can  still  convey  useful  3-D  information.  With 
that  observation  we  turn  to  problems  of  defining  "information  con¬ 
tent."  Finally,  we  discuss  a  computational  methodology  for  studying 
human  vision  that  approaches  complex  systems  at  several  levels  of  de¬ 
tail  or  specificity,  where  different  theoretical  tools  are  appropriate 
at  each  level.  This  approach  has  led  to  succinct  and  precise  descrip¬ 
tions  of  various  processes  of  human  vision  (such  as  stereopsis  and  edge 
detection,  see  later).  That  point  is  crucial  for  its  application  to 
flight  simulation;  even  though  we  currently  have  only  a  limited  under¬ 
standing  of  the  overall  visual  system,  many  facts  that  we  do  know 
translate  into  important  suggestions  for  simulator  scene  generation. 

The  rest  of  the  paper  attempts  to  show  how  that  insight  gomes  about. 


Some  Remarks  Regarding  Training 

The  flight  simulator  that  we  have  in  mind  is  a  device  for  training 
low-level  flight.  One  school  of  thought  is  that  a  training  device 
should  be  optimized  for  the  particular  training  task  for  which  it  is 
intended.  Consequently,  the  distinction  between  training  a  skill  to 
a  novice  and  training  for  performance  or  skill  maintenance  should  be 
reflected  by  the  simulator  system,  for  example.  But  it  is  doubtful  the 
state  of  the  art  of  LLF  simulation  is  sufficiently  advanced  as  to  make 
this  distinction  worthwhile  as  of  yet.  There  are  clearly  two  rather 
diverse  types  of  research  required:  applied  visual  science  in  order 
to  improve  the  effectiveness  of  the  visual  display,  and  training  re¬ 
search  in  order  to  optimize  the  effectiveness  of  the  overall  system  for 
flying  training.  Both  types  of  research  are  needed;  the  question  is  to 
what  extent  they  might  profitably  be  undertaken  independently.  One 
might  argue  that  if  one  attends  solely  to  the  visual  problems  (without 
regard  for  its  role  as  a  device  for  training  pilots)  ,  one  might  inad¬ 
vertently  concentrate  on  visual  problems  that  are  unimportant  to  train¬ 
ing,  or  overlook  problems  that  are  important  to  training.  That  danger 
exists,  but  the  problems  that  beset  LLF  simulation  are  probably  in¬ 
tractable  unless  some  decomposition  is  made  of  the  overall  complex  of 
issues . 

The  pragmatic  view  taken  here  is  that  careful  research  into  the 
generation  of  visual  scenes  (that  leads  to  greater  precision  in  inter¬ 
acting  with  the  environment)  can  proceed  without  much  regard  for  the 
larger  role  of  the  display  in  training.  But  also,  careful  research  is 
needed  regarding  the  training  proper.  For  example,  if  one  is  to  pursue 
optimization  of  the  simulator  for  a  particular  type  of  training  (novices 
versus  skill  maintenance,  say),  it  is  necessary  to  understand  how  the 
two  training  tasks  differ  and  how  the  visual  displays  should  reflect 
those  differences,  in  principle.  But  these  issues  are  also  difficult. 
For  example,  even  in  the  case  of  simulated  landings  (probably  the  best- 
studied  flight  maneuver  for  simulation),  it  is  neither  intuitive  nor 
(to  my  knowledge)  empirically  known  how  to  optimize  the  visual  display 
for  maintenance  versus  undergraduate  training.  Thus  there  are  two 
issues  at  hand:  visual  perception  and  optimal  pilot  training.  Effort 
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on  one  problem  should  not  be  made  at  the  exclusion  of  the  other.  How¬ 
ever,  we  will  not  deal  with  issues  of  training  here. 


Differences  Between  CIG  Simulation  and  Real  Terrain 

When  the  CIG  simulation  of  LLF  is  compared  to  the  real  world  seen 
from  a  low-flying  aircraft,  the  simulation  seems  oversimplified  and 
somewhat  corrupted  by  artifacts,  such  as  the  stuttering  effect  that 
results  when  the  CIG  system  cannot  update  the  display  in  the  periphery 
often  enough  given  the  large  angular  velocities  associated  with  LLF. 

One  might  conclude  that  the  3-D  impression  would  be  adequate  if  only 
one  had  greater  display  capability.  But  it  is  conceivable  that  even  a 
hundredfold  improvement  would  not  solve  the  problem. 

Insufficiency  of  detail  is  not  the  whole  sto?_y.  A  simulation  scene 
is  not  merely  a  simplification  of  a  real  scene;  there  are  dramatic  and 
qualitative  differences.  In  reducing  the  bandwidth  of  a  real  visual 
scene  to  manageable  proportions,  there  is  no  straightforward  sampling 
technique  available — we  cannot  merely  take  every  n-th  bit  of  informa¬ 
tion,  as  it  were.  Instead,  we  construct  a  novel  and  unique  world  out 
of  relatively  few  edges  and  perhaps  shading,  color,  points,  and  lines. 

The  visual  world  we  construct  in  the  simulator  shares  many  simi¬ 
larities  with  the  real  world.  The  single  most  important  similarity  is 
that  the  rules  of  perspective  geometry  are  preserved  in  the  simulation. 
That  means  the  visual  world  in  the  simulation  behaves  optically  as  we 
expect.  This  point  should  not  be  underrated,  for  it  is  responsible 
for  much  of  the  success  of  visual  simulation.2  Specifically,  the  per¬ 
spective  transformation  by  which  the  simulator  CIG  system  projects  3-D 
points,  lines,  and  surfaces  from  the  terrain  model  onto  the  2-D  display 
screen  (and  in  turn,  onto  the  retina  so  long  as  the  display  is  viewed 
from  the  proper  location)  precisely  corresponds  to  the  way  real  3-D 
points  project  onto  the  retina. 3  Consequently,  any  static  view  is 
reproduced  with  geometric  precision  (e.g.,  the  proper  "texture  gradi¬ 
ent"  is  generated  because  incr  asingly  distant  surface  elements  appear 
progressively  smaller).  Also,  if  the  surfaces  are  modeled  as  opaque, 
the  CIG  system  (usually)  has  the  capability  to  make  nearer  surfaces 
occlude  from  view  those  that  are  farther.  Moreover,  if  we  move  in 
space,  the  continuous  changes  in  perspective  of  points  and  surfaces 
also  correspond  geometrically  to  what  happens  in  the  real  world.  Con¬ 
sequently,  we  also  have  geometrically  correct  "optic  flow,”  "motion 
parallax,"  and  so  forth  (see  later).  It  is  the  fact  that  perspective 
projection  is  accurately  duplicated  by  the  CIG  system  that  was 


Here  we  are  suggesting  that  much  of  the  three-dimensionality  comes  by 
having  correct  perspective,  regardless  of  what  is  displayed  in  perspec¬ 
tive.  But  that  is  not  to  say  that  some  benefits  cannot  be  achieved  by 
carefully  crafted  distortions  to  the  perspective  (see  Finch,  1977). 

3 

But  see  Kraft  et  al.  (1980)  for  engineering  considerations. 
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responsible  for  the  compelling  effect  of  space  in  early  night-flight 
simulators,  where  only  moving  luminous  dots  (such  as  runway  lights  and 
city  lights)  were  projected.  As  opaque  surface  shading  was  added  and 
daytime  scenes  were  simulated,  the  additional  gains  came,  in  part,  from 
the  visually  apparent  occlusion  of  distant  objects  by  nearer  ones.  But 
along  with  attempts  to  simulate  daytime  scenes  came  the  realization 
that  the  simulation  is  very  different  from  the  real  thing. 

One  difference  of  probably  minor  importance  is  that  the  display  is 
effectively  a  monocular  presentation  (see  later).  But  just  to  note  a 
few  of  the  profound  differences  between  the  real  scene  and  the  simu¬ 
lated  counterpart,  observe  that  real  surfaces  have  detail  at  all  scales, 
and  that  new  detail  is  continuously  revealed  as  one  approaches  a  sur¬ 
face.  But  the  limited  detail  that  most  CIG  systems  can  display  com¬ 
bined  with  their  resolution  limits  result  in  drastically  simplified 
scenes.  (This  fact  may  be  particularly  relevant  to  the  LLF  question, 
as  pilots  in  real  flight  conceivably  use  fine  recognizable  detail  of 
vegetation  and  rocks  in  order  to  judge  their  distance  from  approaching 
terrain.)  Another  difference  is  that  real  surfaces  are  not  restricted 
in  their  3-D  shape  (consider  a  rolling  desert  floor  or  an  eroded  can¬ 
yon  wall).  But  the  usual  computer  representation  of  curved  surfaces 
in  terms  of  planes  bounded  by  polygons  results  in  a  dramatically  dif¬ 
ferent  sort  of  terrain  than  would  be  found  in  nature.  As  it  is  not 
simply  the  fact  that  the  surface  is  piecewise  planar  rather  than  con¬ 
tinuously  curved,  it  is  also  not  true  that  sharp  straight  boundaries 
occur  at  every  junction.  Those  straight  boundaries  are  also  used  to 
denote  field  boundaries  (such  as  the  checkerboards  which  resemble  agri¬ 
cultural  land).  In  fact,  the  basic  element  in  simulation  for  represent¬ 
ing  surfaces  is  an  edge  across  which  screen  intensity  is  sharply  dis¬ 
continuous.  The  intensity  edge  may  represent  a  sharp  physical  feature 
such  as  a  cliff,  ridge,  or  corner  of  a  building  or  a  place  where  the 
surface  reflectance  sharply  changes,  such  as  the  border  of  a  runway 
or  the  edge  of  a  checkerboard  square.  Some  display  systems  have  shad¬ 
ing  and  modulation  of  contrast  to  approximate  atmospheric  haze  or  fog, 
but  generally  the  intensity  edges  are  equally  sharp  and  homogeneous  in 
CIG  simulations,  and  that  is  not  natural.  Another  point  is  that  actual 
surfaces  have  physical  texture  in  relief  above  the  mean  surface  level 
(bushes  above  the  ground,  peaks  and  troughs  of  waves)  but  this  is  costly 
to  generate  in  current  simulator  systems.  One  final  difference  should 
be  pointed  out:  actual  surfaces  reflect  light  in  a  complex  way  depend¬ 
ing  upon  the  orientation  of  the  light  source  (the  sun  and  the  overall 
sky  illumination),  the  orientation  of  the  viewer  relative  to  the  sur¬ 
face,  and  the  physical  properties  of  the  surface.  But  generally  the 
intensities  of  simulated  surfaces  are  unnaturally  constant  and  homo¬ 
geneous — even  the  most  sophisticated  techniques  for  shading  surfaces 
are  highly  simplified. 

This  discussion  is  not  intended  to  point  out  the  well-appreciated 
fact  that  CIG  scenes  are  unrealistic.  Its  purpose  is  to  show  that 
simulated  terrain  is  an  extreme  and  stylized  simplification  of  the  real 
world.  That  is  not  necessarily  detrimental,  however,  for  human  vision 
has  a  remarkable  ability  to  ignore  simplifications  in  illustrations. 

For  instance,  we  are  all  familiar  with  the  strong  visual  impact  that 
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even  simple  line  drawings  provide,  such  as  seen  in  engineering  and 
mathematics  texts  and  in  assembly  instructions.  While  they  are  highly 
unrealistic,  we  seldom  attend  to  that  fact.  Since  they  carry  the 
necessary  3-D  information,  they  serve  their  purpose;  it  does  not  mat¬ 
ter  that  actual  surfaces  give  rise  to  more  complicated  images.  Simi¬ 
larly,  it  may  be  argued  that  simulation  displays  of  terrain  may  be 
adequate  (for  purposes  of  training,  etc.)  and  yet  highly  unrealistic 
and  caricatured. 


Problems  with  Intuitive  and  Geometric  Argument 

Thus  we  have  that  the  C1G  display  may  carry  the  necessary  3-D  in¬ 
formation  despite  the  unrealistic  qualities  just  described.  But  we 
still  have  to  formalize  what  we  mean  by  "necessary  3-D  information." 

In  that  pursuit  it  should  be  stressed  that  our  intuition  should  not  be 
trusted.  Intuition  is  often  wrong  regarding  the  way  the  human  visual 
system  operates.  In  short,  what  we  naively  believe  to  govern  our 
visual  perceptions  is  often  not  the  case.  It  is  worthwhile  examining 
one  such  example  in  detail.  The  example  involves  the  familiar  (static) 
texture  gradient,  specifically,  that  the  texture  density  seems  intui¬ 
tively  to  be  the  crucial  depth  cue  in  texture  gradients  the  higher  the 
density,  the  greater  the  distance  to  the  surface  (Gibson,  1950);  see 
also  the  geometric  analysis  in  Purdy  (1960).  In  fact,  distance  cannot 
be  inferred  directly  from  density  (the  following  argument  is  summarized 
from  Stevens  [1980]).  The  reason  is  that  texture  density  is  a  function 
not  only  of  the  distance  to  the  surface  but  also  of  the  slant  of  the 
surface  relative  to  the  viewer  (the  greater  the  slant,  the  greater  the 
foreshortening  and  hence  the  greater  the  texture  density).  In  the  case 
of  an  arbitrary  surface,  one  cannot  decouple  the  relative  contributions 
to  the  density  gradient  caused  by  foreshortening  from  that  due  to  dis¬ 
tance.  Consequently,  one  cannot  infer  distance  from  texture  density. 
This  argument  provides  an  explanation  for  the  largely  ignored  psycho¬ 
physical  evidence  (e.g..  Smith  and  Smith,  1957;  Braunstein,  1968; 
Braunstein  and  Payne,  1969)  that  texture  density  is  an  ineffective  cue 
to  distance.  This  result  is  contrary  to  our  intuitions.  It  points 
out  that  we  must  go  beyond  introspection. 

In  the  same  vein,  it  is  not  sufficient  merely  to  examine  che  geo¬ 
metric  properties  of  perspective  projection.  For  instance,  the  analy¬ 
sis  of  texture  gradients  that  Purdy  (1960)  performed  set  out  to  find  a 
geometric  basis  for  the  hypotheses  that  Gibson  (1950)  set  forth  (such 
as  the  one  just  discussed).  But  such  geometric  analyses  must  be  care¬ 
fully  regarded,  as  they  implicitly  embody  certain  geometric  assump¬ 
tions,  such  as  that  the  ground  is  globally  planar  (see  Purdy,  1960). 

The  mathematical  relations  derived  with  those  assumptions,  of  course, 
do  not  hold  when  the  physical  surfaces  are  not  so  constrained  (e.g., 
when  the  textured  surface  is  not  planar).  It  is  not  a  straightforward 
matter  to  set  down  the  geometric  relations  that  provid  the  basis  for 
visual  interpretation  of  the  image.  The  relations  necessarily  incor¬ 
porate  constraints,  and  it  is  a  separate  and  nontrivial  matter  to 
verify  that  those  constraints  are  adopted  by  the  actual  perceptual 
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processes  of  human  vision.  For  further  discussion  on  the  matter  of 
constraints  and  its  relation  to  Gibson's  "direct  perception"  hypothe¬ 
sis,  see  Ullman  (1980). 


How  Is  Information  Encoded  in  the  Image? 

Image  information  content  is  usually  discussed  in  terms  of  depth 
cues  such  as  texture  gradients,  optic  flow,  and  stereopsis.  But  to  be 
useful,  in  particular  to  be  useful  for  flight  simulation,  those  terms 
must  be  defined  more  precisely  than  is  usual  in  the  psychological  lit¬ 
erature  (see  also  Hennessy  et  al.,  1980;  Semple  et  al.,  1980).  To 
illustrate,  consider  the  difficulty  a  simulator  designer  would  have 
trying  to  apply  a  general  fact  about  optic  flow.  For  argument,  sup¬ 
pose  he  reads  that  optic  flow  provides  information  about  the  direction 
of  travel  because  of  the  streaming  of  detail  radially  away  from  the 
point  he  is  approaching  (Gibson,  1950).  If  the  designer  learns  that 
pilots  often  lose  visual  orientation  in  the  simulator  (while  maneuver¬ 
ing,  say),  the  designer  would  recall  that  optic  flow  is  relevant.  But 
the  designer  is  now  stuck,  in  a  word,  because  optic  flow  was  not  de¬ 
fined  with  sufficient  specificity  that  the  designer  might  translate 
that  suggestion  into  an  improved  visual  display.  It  is  a  rather  subtle 
point  that  optic  flow  is  inevitable  whenever  one  moves  through  other 
than  empty  space.  Optic  flow  is  a  consequence  of  perspective  projection 
and  motion,  so  regardless  of  how  the  terrain  is  depicted,  if  one  moves 
across  it,  the  resulting  movement  of  detail  across  the  visual  field  is 
optic  flow. 

The  designer  is  faced  with  a  terrain  simulation  that  is  apparently 
inadequate,  seeks  to  improve  it,  and  knows  that  optic  flow  is  relevant. 
So  he  considers  the  variables  that  govern  optic  flow,  those  being  the 
parameters  of  motion  relative  to  the  surface  as  well  as  the  qualities 
of  the  surface  texture  itself.  (The  faster  and  lower  the  aircraft,  the 
more  pronounced  the  apparent  optic  flow;  also,  the  more  densely  tex¬ 
tured  the  surface,  the  greater  the  effect.)  Of  these  variables,  it  is 
only  the  surface  texture  that  the  designer  may  manipulate,  but  how  to 
do  so?  Insight  would  come  from  knowing  something  about  how  the  human 
visual  system  measures  and  internally  encodes  moving  texture  and  how  it 
extracts  3-D  information  such  as  orientation  from  those  measurements. 

We  cannot  dismiss  this  issue  in  a  cavalier  manner,  trusting  that  the 
human  visual  system  will  extract  what  it  needs  so  long  as  the  CIG  dis¬ 
play  is  rich  enough  in  detail.  Current  (and  foreseeable)  CIG  display 
technology  does  not  (and  probably  will  not)  have  the  capability  of 
reproducing  the  complex  visual  texture  presented  by  real  LLF  over 
natural  terrain.  Rather,  we  should  devise  more  efficient  uses  of  the 
CIG  system's  ability  to  generate  texture.  We  might  think  of  this  as 
"impedance  matching,"  of  matching  the  type  of  texture  that  is  displayed 
with  the  type  of  visual  processing  we  make  on  that  texture,  so  as  to 
optimize  the  information  transfer.  For  instance,  the  visual  display 
might  concentrate  its  detail  at  the  (instantaneous)  point  of  gaze  and 
capitalize  on  the  fact  that  visual  resolution  degrades  with 
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eccentricity.^1  But  much  more  thought  is  needed;  it  is  not  enough  to 
discuss  what  might  be  omitted  (such  as  detail  in  the  periphery)  but 
what  might  be  included  in  the  texture.  Such  thought  requires  that  one 
examine  the  visual  processes  in  terms  more  analytical  than  simply 
"depth  cues." 


A  Computational  Information  Processing  Approach 

We  now  sketch  an  approach  to  the  study  of  vision  which  has  appli¬ 
cation  to  visual  simulation.  It  is  probably  well  to  remark  at  the 
outset  that  this  approach  stands  among  many  approaches  to  vision  re¬ 
search.  Later  we  will  see  how  this  approach,  because  of  its  different 
perspective,  emphasizes  certain  issues  of  terrain  depiction  that  have 
not  been  addressed  previously  in  simulation  research. 

It  has  long  been  regarded  that  vision  embodies  processes  that  de¬ 
rive  information  about  the  visual  world  from  the  interpretation  of 
retinal  images.  To  be  sure,  virtually  every  phrase  in  the  previous 
statement  has  been  contested,  cast  in  a  different  light,  or  relatively 
emphasized  by  a  particular  theoretical  viewpoint.  For  instance, 

Gibson  (1950,  1979)  characterizes  human  vision  as  direct  registration 
of  higher-order  variables  in  the  visual  stimuli,  thereby  downplaying 
the  need  for  interpretation  and  information  processing  (but  see  Ullman, 
1979).  There  are  some  conflicting  theories  of  visual  processing,  but 
largely  the  differences  are  of  emphasis.  For  instance,  some  research¬ 
ers  concentrate  on  the  fact  that  our  perception  is  often  as  much  a 
product  of  what  we  expect  as  what  is  objectively  present  in  the  image. 
Others  concentrate  on  the  fact  that  natural  scenes  have  richly  redundant 
and  consistent  sources  of  information,  and  argue  that  since  human  vision 
probably  capitalizes  on  that  fact,  experiments  using  simplified  stimuli 
must  be  carefully  interpreted. 

Another  theoretical  viewpoint  is  exemplified  by  the  computational 
approach  of  Marr  (1976,  1978,  1981)  and  Marr  and  Poggio  (1977),  which 
examines  the  information  processing  aspects  of  vision.  The  value  of 
this  approach  to  vision  research  stems  largely  from  recognizing  (i)  that 
human  vision  involves  complex  information  processing,  and  (ii)  that 
complex  systems  are  feasibly  understood  by  us  only  when  described  at 
several  levels  of  detail  and  abstraction.  The  first  point  is  well  ac¬ 
cepted  by  psychologists  (in  fact,  some  researchers  find  the  complexity 
overwhelming  and  believe  we  will  never  understand  the  processes  of 
vision  because  of  their  complexity) .  The  second  point  is  well  accepted 
by  engineers,  and  particularly  computer  scientists,  who  see  the  need 
for  making  clean  distinctions  between  the  purposes  or  goals  of  a  com¬ 
plex  device,  the  methods  by  which  it  achieves  those  goals,  and  the  par¬ 
ticular  nuts-and-bolts  details  of  how  they  are  carried  out.  The  vo¬ 
cabulary  used  for  describing  what  a  system  accomplishes  is  very  different 


One  should  be  careful  here,  because  while  resolution  degrades  with  ec¬ 
centricity,  temporal  processing  is  still  strong  (e.g.,  llicker  and 
motion  are  readily  detected  in  the  periphery) . 
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than  that  used  for  describing  how  it  does  that  (at  various  levels  of 
detail,  culminating  in  the  physics  of  transistors,  or  whatever  com¬ 
prises  the  basic  building  blocks  of  the  system) .  Complex  information 
processing  systems  require  this  sort  of  multilevel  description;  any 
one  level  of  description,  alone,  would  be  inadequate.  For  instance, 
the  circuit  diagram  of  a  digital  computer  at  the  level  of  individual 
transistors  and  capacitors  would  be  incomprehensible,  if  it  were  not 
for  additional  descriptions  of  how  the  electronic  components  implement 
logical  functions  such  as  "and”  and  "or,"  and  how  those  functions  are 
sequenced  to  achieve  processes,  and  so  forth.  Marr  argues  that  bio¬ 
logical  information  processing  should  be  similarly  approached.  Neuro¬ 
physiology  and  anatomy  provide  us  some  understanding  of  the  detailed 
architecture  of  the  visual  system,  but  just  as  studying  the  physics  of 
a  transistor  reveals  nothing  about  operating  systems  or  Fortran  com¬ 
pilers,  neither  does  study  of  neural  synapses  reveal  the  principles  of 
stereopsis  or  motion  perception. 

This  computational  approach  describes  visual  processes  at  several 
levels. 5  Ideally,  one  would  like  to  understand  some  visual  process 
(such  as  stereopsis)  at  all  levels,  from  the  abstract  and  mathematical 
down  to  the  level  of  neural  implementation.  But  it  is  only  for  certain 
very  early  stages  of  visual  processing  (such  as  retinal  function)  that 
neurophysiological  details  are  feasibly  incorporated  into  a  computa¬ 
tional  theory  as  of  yet  (see  Marr  and  Hildreth,  1980;  Richter  and  Ull- 
man ,  1980).  Of  course,  one  can  have  a  significant  and  useful  under¬ 
standing  of  a  complex  mechanism  without  knowing  its  implementation 
details.  By  and  large,  the  theories  emerge  "top-down,"  with  the  ini¬ 
tial  insights  gained  at  the  level  of  what  the  visual  system  computes 
(regarding,  for  instance,  stereopsis)  and  the  theoretical  basis  for 
that  computation.  This  sort  of  research  is  reminiscent  of  Gibson  and 
his  followers,  for  one  is  concerned  with  geometric  constraints  and 
feasibility.  But,  in  contrast,  the  computational  approach  concerns 
itself  with  the  form  in  which  visual  information  is  encoded  or  made 
explicit  within  the  visual  system.  The  3-D  representations  are  de¬ 
scribed  abstractly,  not  at  the  neural  level  (that  sort  of  understand¬ 
ing  is  far  from  us  at  the  present)  (Marr,  1976,  1978;  Marr  and  Nishi- 
hara,  1978;  Stevens,  1980). 

In  formalizing  the  computations  that  underlie  vision,  one  makes 
rigorous  the  form  in  which  the  visual  information  is  made  explicit  by 
the  visual  system,  the  sources  of  that  information  in  the  image,  how 
that  information  is  actually  extracted  and  measured,  and  finally  the 
computational  constraints  necessary  to  interpret  that  information. 

This  sort  of  vision  research  has  relevance  to  simulation,  therefore, 
because  understanding  how  information  is  extracted  tells  us  how  it 


See  Marr  and  Hildreth  (1980)  and  Richter  and  Ullman  (1980)  for  de¬ 
tailed  theories  of  retinal  processing  and  edge  detection.  See  Marr 
and  Poggio  (1979),  Mayhew  and  Frisby  (1981),  and  Grimson  (1980)  for 
computational  theories  of  stereopsis.  See  Ullman  (1979)  for  visual 
motion,  and  Stevens  (1980,  1981)  for  texture  gradients  and  surface 
contours . 


46 


should  be  presented.  Also,  the  constraints  that  visual  processes  in¬ 
corporate  are  essentially  perceptual  assumptions  about  the  nature  of 
the  visual  world.  Those  assumptions  are  either  always  valid,  because 
of  the  physics  of  solid  objects,  perspective,  and  optics,  or  the  as¬ 
sumptions  are  usually  valid  because  of  the  statistical  properties  of 
the  world;  see  Ullman  (1979)  for  discussion  of  assumptions  in  computa¬ 
tional  terms  and  Gibson  (1979)  for  the  related  notion  of  "ecological 
optics."  These  assumptions  have  relevance  to  simulation,  for  in  laying 
out  a  simulated  3-D  environment  it  is  important  to  have  its  properties 
match  our  visual  assumptions;  otherwise,  our  perceptual  reconstruction 
of  that  environment  will  not  correspond  to  what  was  intended.  A  spe¬ 
cific  example  will  be  given  concerning  the  assumptions  underlying  our 
interpretation  of  surface  contours.  Those  results  are  primarily  geo¬ 
metric;  other  relevant  contributions  to  simulator  research  pertain  to 
efficient  depictions  of  texture. 

The  rest  of  the  paper  reflects  on  the  problems  of  depicting  ter¬ 
rain  for  flight  simulation  from  the  computational  perspective. 


Shape,  Orientation,  and  Scale 

Let  us  consider  in  general  terms  what  the  pilot’s  visual  system 
must  accomplish  in  order  that  an  aircraft  may  be  maneuvered  at  low  al¬ 
titude  over  actual  terrain.  Of  course,  the  shape  of  the  terrain  must 
be  perceived  in  3-D,  so  that  a  close  earth-hugging  course  might  be  fol¬ 
lowed.  But  not  merely  the  shape  must  be  known,  but  also  its  size  or 
scale.  A  small  hill  and  a  mountain  might  have  the  same  shape  but  dif¬ 
fer  significantly  in  scale.  We  will  need  to  become  more  specific  about 
shape  and  scale,  but  let  us  continue  informally  for  a  moment.  One  must 
also  know  the  pilot's  orientation  and  altitude  relative  to  the  terrain. 
The  pilot,  it  is  reported,  envisions  himself  as  following  a  path  through 
space  above  the  ground,  where  he  keeps  a  mental  trace  of  where  he  has 
been,  where  he  is,  and  a  projection  in  front  of  him  of  where  he  is  going. 
In  large  part  this  must  come  from  his  visually  perceiving  his  orienta¬ 
tion  relative  to  the  terrain.  We  must  also  be  specific  about  what  this 
entails . 

We  have  singled  out  shape,  scale,  and  orientation  as  three  classes 
of  visual  information  necessary  for  flying  over  terrain.  Although  we 
are  being  informal  in  our  use  of  the  terms,  these  three  notions  seem 
amenable  to  a  precise  definition  eventually.  But  is  this  decomposition 
of  the  visual  requirements  for  LLF  into  the  perception  of  shape,  scale, 
and  orientation  the  best  decomposition  for  purposes  of  improving  the 
visual  simulation?  It  certainly  seems  that  each  form  of  information  is 
basic  and  necessary.  On  the  other  hand,  concern  is  perhaps  warranted 
that  the  shape,  scale,  and  orientation  are  not  sufficiently  inclusive; 
we  might  be  omitting  some  different  quality  of  3-D  information  which  is 
necessary  for  LLF.  (Note  that  we  are  not  considering  information  about 
tactical  targets,  navigation  cues,  and  so  forth.  Rather,  we  are  solely 
concerned  with  the  information  that  must  be  gathered  by  the  visual  sys¬ 
tem  so  that  the  pilot  might  fly  just  above  the  terrain.)  We  do  not  know 
the  answer  to  that  question;  nonetheless,  the  shape-orientation-scale 
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approach  to  describing  necessary  information  should  prove  useful  even 
if  it  is  not  the  "whole  story." 

The  distinction  between  shape  and  scale  is  important.  Human  vision 
is  often  surprisingly  imprecise  about  scale:  we  appreciate  the  shape  of 
a  microorganism  from  a  scanning  electron  micrograph  without  any  concern 
for  (or  appreciation  of)  its  size;  likewise,  an  astrophotograph  of  a 
nebulus  imparts  in  us  no  sense  of  scale.  This  phenomenon  is  not  diffi¬ 
cult  to  explain  theoretically ;  it  is  likely  that  the  internal  repre¬ 
sentations  of  visual  shape  are  inherently  scale-independent  (see  the 
notion  of  scales  in  Haber  and  Hershenson  (1980)  and  the  representations 
of  surface  orientation  in  Attneave  (1972)  and  Marr  (1978).  Furthermore, 
the  visual  system  probably  treats  as  distinct  perceptual  problems  the 
determination  of  an  object’s  shape  and  size.  This  distinction  between 
shape  and  scale  is  often  nonintuitive  because  in  everyday  scenes  we 
usually  have  rich  sources  to  both  forms  of  information;  there  are  many 
convergent  perceptual  processes  that  provide  information  about  shape, 
and  there  are  many  means  by  which  we  directly  and  indirectly  determine 
the  actual  size  and  distance  of  visible  objects.  The  scale  of  our  im¬ 
mediate  surrounding  area  is  usually  so  precisely  known  that  we  interact 
in  it  with  grace  and  facility.  But  we  are  also  faced  with  natural  situ¬ 
ations  in  which  the  scale  information  is  relatively  impoverished.  One 
case  with  which  most  of  us  are  familiar  concerns  viewing  the  earth  from 
the  air,  where  the  shape  of  the  mountains  or  canyons  is  clear  to  us, 
but  their  size  is  appreciated  only  if,  say,  we  detect  a  cabin,  or  a  car 
on  a  road.  We  then  experience  a  sudden  and  sometimes  shocking  appreci¬ 
ation  6 f  the  scale  of  the  scene  and,  simultaneously,  of  the  actual  dis¬ 
tances  involved.  (The  apparent  shape  remains  unchanged;  however,  after 
we  learn  the  scale,  it  is  as  if  independent  information  is  added  to 
our  perception.)  The  crucial  points  we  should  draw  are  two:  (i)  scale 
and  shape  are  distinct  forms  of  information  about  the  3-D  world,  and 
(ii)  both  forms  of  information  are  needed  in  order  to  interact  precisely 
with  the  w'prld. 

The  third  class  of  3-D  information,  orientation,  has  many  facets 
and  is  closely  intertwined  with  shape.  First  we  will  discuss  what  we 
mean  by  orientation  and  then  show  how  shape  and  orientation  relate. 
Gibson  (1950)  discusses  two  qualities  of  orientation;  one  local,  the 
other  global.  The  local  orientation  of  surfaces  is  defined  relative  to 
the  viewer;  one’s  orientation  in  space  is  defined  globally  relative  to 
the  surrounding  visual  environment.  Let  us  consider  each  in  turn.  It 
is  natural  to  visualize  the  orientation  of  a  patch  of  surface  relative 
to  oneself.  The  slant  angle  between  the  line  of  sight  and  the  normal 
to  the  surface  is  one  way  of  quantifying  this.  When  viewing  a  large 
planar  surface  such  as  the  ground  seen  from  the  air,  there  is  a  wide 
range  of  relative  surface  orientation  (zero  slant  directly  below  the 
aircraft,  slant  approaching  90  degrees  toward  the  horizon).  But  we 
think  of  the  planar  surface  as  having  one  orientation,  not  merely  a 
local  orientation  that  depends  on  which  patch  of  the  surface  we  view. 

If  the  plane  rolls,  we  see  the  orientation  of  the  ground  change  relative 
to  us.  We  clearly  have  the  ability  to  judge  orientation  more  globally, 
e.g.,  relative  to  the  horizon. 


We  hav<  discussed  two  sorts  of  orientation  information:  informa¬ 
tion  about  the  attitude  of  individual  patches  of  surfaces  relative  to 
the  viewer,  and  the  overall  orientation  of  a  large  surface  such  as  the 
ground  seen  from  the  air.  There  is  another  related  sort  of  information 
which  also  seems  primitive  and  important:  the  direction  of  movement. 
Whether  running  through  a  forest  or  flying  at  low  altitude,  we  are 
visually  aware  of  which  way  we  are  traveling  relative  to  the 
environment . 

If  we  regard  shape,  orientation,  and  scale  as  three  types  of  3-D 
information  necessary  for  flying  relative  to  the  terrain,  we  may  ap¬ 
proach  the  problems  of  improving  the  terrain  simulation  in  these  terms. 
The  approach  is  as  follows. 

The  Basic  Approach 

The  visual  deficiencies  of  the  LLF  simulation  would  be  cast  in 
terms  of  shape,  orientation,  and  scale.  In  other  words,  one  would  as¬ 
certain  where  the  performance  problems  lie  in  simulated  LLF  flight,  to 
what  extent  the  cause  is  inadequate  perception  of  the  shape  of  the  ter¬ 
rain  (Is  that  ridge  sharp  or  is  it  smoothly  rounded?  What  does  the 
terrain  do  over  to  the  right?),  inadequate  perception  of  orientation 
(Is  the  aircraft  yawing?  Is  the  ground  ahead  rising  or  horizontal?)  , 
and  inadequate  perception  of  scale  (How  high  am  I  above  the  ground? 

How  far  is  the  ridge?).  Insight  into  the  underlying  problems  cast  in 
these  terms  might  be  gained  by  many  means.  Pilots  might  even  be  ver¬ 
bally  examined  in  these  terms,  because  we  all  have  a  strong  (and  simi¬ 
lar)  understanding  of  what  is  meant  by  shape,  orientation,  and  scale. 

It  was  mentioned  earlier  that  pilots  commonly  remark  that  judging  al¬ 
titude  is  difficult.  That,  for  instance,  is  a  matter  of  scale. 

In  addition  to  direct  interview,  there  should  be  more  quantitative 
psychophysical  means  for  determining  what  3-D  information  is  deficient, 
but  again,  cast  only  in  terms  of  shape,  orientation,  and  scale,  without 
attempting  to  uncover  what  "depth  cues"  are  missing  from  the  CIG  display. 

Note  that  we  seem  to  attack  what  has  been  a  thorny  problem  with 
only  very  informal  and  casual  terminology.  But  the  rigor  comes  later, 
when  we  apply  knowledge  about  visual  perception  to  suggest  improvements 
to  scale,  or  whatever.  We  will  see  an  example  of  this  analysis  when 
we  discuss  how  shape  is  perceived  from  undulating  surface  contours. 

This  would  have  importance  in  the  CIG  depiction  of  rolling  terrain. 

First  it  will  be  useful  to  examine  a  wide  range  of  visual  processes 
so  that  we  begin  to  sort  out  which  processes  concern  shape,  which  con¬ 
cern  orientation,  and  so  forth.  In  a  sense,  we  are  abandoning  the 
rather  simple  notion  of  "depth  cue"  for  three  types  of  surface  infor¬ 
mation,  not  just  depth.  This  has  proven  useful  in  vision  research 
proper.  To  illustrate,  stereopsis  is  not  merely  a  "cue"  to  depth  but 
also  of  shape,  at  least;  shading  also  provides  shape  information  but 
tells  us  nothing  about  scale  (the  shading  of  the  moon  is  similar  to 
that  of  an  orange) .  We  cannot  expect  a  simple  correspondence  between 
process  and  either  shape,  or  orientation,  or  scale — vision  is  far  too 
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intertwined  to  be  cast  in  such  simple  terms.  But  on  the  other  hand,  if 
we  know  that  we  need  additional  information  about  shape,  we  have  many 
visual  means  for  providing  that  information.  It  goes  without  saying 
that  our  understanding  of  vision  is  not  sufficiently  complete  as  to 
provide  ready-made  recipes  for  improving  the  simulation. 

As  a  final  introductory  remark,  note  that  although  we  might  treat 
scale,  orientation,  and  shape  as  substantially  independent  topics,  we 
will  probably  find  that  improvements  to  the  CIG  display  intended  to 
enhance,  say,  the  perception  of  shape  will  also  improve  the  apparent 
orientation,  and  so  forth.  These  side  effects  would  be  welcomed,  of 
course.  But  the  general  strategy  is  to  understand  how  one  gains  a 
perception  of  shape,  orientation,  and  scale  in  order  to  increase  the 
effectiveness  of  the  simulator  in  inducing  those  perceptions. 


Shape:  What  Are  the  Sources? 

There  are  several  potential  sources  of  information  about  the  shape 
of  a  surface,  including  binocular  disparity  from  stereopsis,  visual  mo¬ 
tion,  shading,  texture  gradients,  and  surface  contours.  Let  us  consider 
each  in  turn. 

First,  although  stereopsis  provides  an  acute  sense  for  3-D  shape 
in  our  usual  environment,  it  is  not  useful  for  large-scale  shape  per¬ 
ception,  e.g.,  of  terrain  a  few  miles  away.  The  reason  is  that  only 
the  very  near  terrain  causes  detectable  stereo  disparities,  and  since 
that  portion  of  the  scene  is  also  moving  across  the  field  of  view  at 
high  angular  velocity,  one  must  track  some  nearby  surface  feature; 
otherwise  it  is  blurred.  On  the  other  hand,  stereopsis  cannot  be  ruled 
out  as  potentially  determining  scale ,  especially  in  the  foreground  (see 
later)  . 

Motion:  Parallax,  Optic  Flow,  and  Shear  Visual  motion  provides 
one  of  the  dominant  sources  of  information  about  shape.  Several  differ¬ 
ent  aspects  of  motion  have  been  distinguished,  such  as  "shear"  (dis¬ 
continuities  in  projected  angular  velocity  which  arise,  for  example, 
along  the  edge  of  a  physical  object  seen  against  a  relatively  distant 
background)  ,  "optic  flow"  (the  wide-field  visual  effect  from  movement 
through  the  environment)  ,  and  "motion  parallax"  (the  changing  projec¬ 
tion  of  an  object  as  it  moves  relative  to  us,  from  which  we  infer  its 
3-D  shape) . 

Before  discussing  these  aspects  of  motion,  we  reiterate  that 
visual  motion  across  the  CIG  display  comes  automatically  when  the  3-D 
model  is  transformed  by  perspective  projection  while  "continuously" 
(i.e.,  at  a  sufficient  rate)  updating  the  viewer's  position  in  that 
model  according  the  speed  and  direction  of  motion  relative  to  the 
model.  Thus  motion  per  se  is  given;  it  is  what  is  in  motion  across 
the  CIG  display  that  is  relevant  to  our  discussion.  For  instance,  are 
moving  dots  sufficient?  (Luminous  dots  are  particularly  "cheap"  to 
display  on  many  CIG  systems,  whereas  surface  patches  are  in  relatively 
short  supply.) 
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In  fact,  moving  dots  are  effective:  if  a  collection  of  3-D 
points  in  some  rigid  arrangement  (such  as  the  runway  marker  lights 
and  the  lights  of  nearby  buildings  and  streets)  are  projected  in  mo¬ 
tion  on  the  display  as  if  we  were  moving  relative  to  them  (as  in 
landing),  the  visual  interpretation  in  3-D  is  remarkably  precise. ^ 

Of  course,  the  designers  of  visual  simulator  night  displays  have  long 
been  familiar  with  this.  Note  the  3-D  shape  is  accurately  perceived, 
but  not  the  scale,  unless  there  are  familiar  cues  such  as  the  runway 
width  (see  later) . 

So  we  know  that  points  are  sufficient  stimuli,  but  what  about 
lines  and  edges?  What  do  they  contribute  over  mere  points  of  light? 
This  perceptual  issue  has  not  been  fully  unraveled,  but  some  relevant 
observations  may  be  made.  Our  visual  processes  seem  to  require  dis¬ 
tinct  and  traceable  image  points  in  order  to  derive  a  3-D  shape,  but 
a  curve  or  line  only  offers  the  endpoints  or  points  of  discontinuity 
in  tangent;  arbitrary  intermediate  points  along  the  curve  cannot  be 
tracked  (see  Ullman ,  1979).  Incidentally,  this  suggests  that  only 
the  corner  points  of  the  checkerboard  pattern  (popularly  used  to  de¬ 
pict  terrain)  contribute  useful  input  to  the  motion  interpretation 
process . ? 

When  traveling  across  a  textured  surface  there  is  an  apparent 
streaming  of  detail  across  the  visual  field,  commonly  termed  optic 
flow.  This  is  regarded  as  a  wide-field  phenomenon,  while  the  previ¬ 
ously  discussed  motion  parallax  is  detailed  and  foveal.  What  3-D  in¬ 
formation  is  derived  from  optic  flow  in  the  periphery?  Some  hypotheses 
have  been  forwarded  that  the  optic  flow  specifies  distance  up  to  a 
scalar  (Gibson,  1950;  Nakayama  and  Loomis,  1974)  and  even  local  surface 
orientation  everywhere  across  the  image  (Koenderink  and  van  Doom, 

1976)  .  Those  two  proposals  suggest  that  optic  flow  provides  shape  (but 
not  scale)  information,  but  they  are  largely  theoretical;  it  remains  to 
be  seen  what  sort  of  3-D  information  is  derived,  in  fact,  by  the  human 
visual  system.  It  is  possible  that  negligible  shape  information  is  de¬ 
rived  from  the  periphery,  even  under  the  best  conditions.  Optic  flow, 
particularly  in  the  case  of  LLF,  may  provide  only  information  about 
orientation  and  the  direction  of  travel  (this  is  discussed  further 
later)  . 

Another  aspect  of  visual  motion  is  motion  shear.  Whenever  an 
opaque  surface  feature  protrudes  above  the  mean  surface  level,  it  oc¬ 
cludes  from  view  that  which  is  behind  it.  Occlusion  thereby  provides 


See  Ullman  (1979)  for  a  theorem  showing  how  this  interpretation  is 
feasible  solely  by  analysis  of  the  images  without  requiring  higher- 
level  knowledge. 

^But  that  does  not  mean  one  can  replace  the  checkerboard  with  dots 
(placed  where  each  corner  was)  and  have  an  equally  compelling  3-D 
impression.  The  checkerboard  also  provides  a  useful  texture  gradient, 
at  least.  This  demonstrates  a  difficult  aspect  of  this  work:  visual 
processes  and  their  visual  inputs  are  largely  intertwined. 


information  about  relative  distance — it  allows  us  to  partition  the 
visual  world  into  surfaces  segregated  by  distance.  This  is  particu¬ 
larly  striking  in  LLF  when  the  relative  motion  of  nearby  ridges  against 
the  background  terrain  causes  them  to  be  seen  in  relief.  Since  shear 
contributes  to  the  apparent  terrain  shape,  it  deserves  study.  In  par¬ 
ticular,  the  questions  of  what  is  shearing  and  whether  it  is  a  simple 
dot  pattern  or  edges  or  lines  is  of  importance  to  C1G  design.  The 
perception  of  3-D  shape  from  motion  was  only  briefly  sketched  here. 

It  is  hoped  that  it  shows  that  while  studying  the  perceptual  processes 
one  can  also  derive  specific  implications  for  CIG  display. 

Shading,  Texture  Gradients,  and  Contours  We  have  considered  the 
dynamic  aspects  of  the  visual  display.  The  other  side  of  the  coin  is 
the  static  properties  which  carry  information  about  shape.  These  in¬ 
clude  shading,  texture  gradients,  and  contours  that  lie  across  the 
surface . 

The  apparent  3-D  shape  of  a  surface  is  usually  enhanced  when 
shaded.  The  interpretation  of  shading  information  is  analytically  a 
very  difficult  problem  requiring  knowledge  of  the  illuminant  directions, 
the  reflectance  functions  of  the  various  surfaces,  and  means  for  dealing 
with  complex  shadowing  and  mutual  illumination  conditions  (Horn,  1973). 
But  the  human  visual  system  probably  does  not  attempt  such  a  solution; 
instead,  it  extracts  weaker  inferences  of  3-D  shape  that  would  be  true 
without  having  to  know  the  lumination  conditions  and  the  particular 
reflectance  properties  of  the  surface.  This  probably  explains  the  suc¬ 
cess  of  even  crude  approximations  to  real  shading  that  are  adopted  by 
CIG  systems.  Further  research  into  human  perception  of  shading  should 
lead  to  an  understanding  of  just  how  simple  an  approximation  to  shading 
can  be  effective  in  the  simulator  display. 

Another  source  of  3-D  information,  which  we  will  discuss  only 
briefly,  is  the  texture  gradient .  A  homogeneous  distribution  of  physi¬ 
cal  texture  across  a  surface  results  in  a  texture  gradient  in  the  image. 
Because  of  perspective  projection,  the  image  texture  in  every  locality 
is  both  foreshortened  and  scaled.  We  can  recover  shape  information 
(specifically  distance  up  to  a  scalar)  from  the  texture  gradient,  but 
as  discussed  earlier,  probably  not  from  texture  density.  Further  re¬ 
search  is  needed  to  determine  how  texture  gradients  are  measured  in 
order  that  we  may  gain  insight  into  how  best  to  depict  texture  for  CIG 
display . 

Contours  that  lie  across  a  surface,  as  depicted  in  figure  B-la,  are 
useful  for  inferring  the  shape  of  the  surface.  To  make  sense  of  the 
contours  in  the  image,  however,  we  must  make  certain  assumptions  about 
their  geometry;  see  Stevens  (1981)  for  the  theory  of  surface  contour 
sketched  here.  We  will  turn  this  to  our  advantage  by  ensuring  that 
those  assumptions  are  not  violated  when  modeling  terrain  by  means  of 
contours . 

Observe  in  figure  B-la  that  the  lines  appear  drawn  across  an  undu¬ 
lating  surface.  The  basic  assumption  that  we  make  is  that  the  curvature 
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of  the  lines  reflects  surface  curvature.®  It  is  as  if  the  lines  were 
straight  lines  on  a  carpet  that  became  curved  because  the  carpet  was 
rippled.  More  formally,  these  contours  are  geodesics  with  the  addi¬ 
tional  property  of  being  lines  of  greatest  curvature  (see  Hilbert  and 
Cohn-Vossen,  1952,  for  a  lucid  discussion  of  these  concepts). 


FIGURE  B-l  The  curved  surface  in  (a)  resembles  a  rippled  carpet.  The 
straight  lines  lie  parallel  to  the  ridge,  the  curved  lines 
are  perpendicular.  The  curved  lines  are  lines  of  greatest 
curvature  and  planar;  the  straight  lines  are  lines  of  least 
curvature.  Note  that  the  curved  lines  alone  tell  us  a 
great  deal  about  the  shape  of  the  surface  (b),  but  the 
straight  lines  alone  carry  virtually  no  information. 


A  line  of  greatest  curvature  may  be  thought  of  as  a  path  across  a 
surface  which  experiences  the  greatest  undulation.  Note  that  in  fig¬ 
ure  B-lb  the  straight  lines  that  follow  the  ridges  are  perpendicular  to 


g 

Note  that  we  do  not  easily  see  figure  B-l  for  what  it  is,  a  collection 
of  lines  on  a  flat  sheet  of  paper.  Instead,  we  interpret  the  undulation 
as  having  been  caused  by  the  underlying  surface. 
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the  lines  of  greatest  curvature.  The  straight  lines  are  lines  of 
least  curvature  and  the  surface  is  singly  curved.  The  perpendicularity 
of  the  two  curves  is  very  important — it  gives  us  constraint  on  the 
shape  of  the  surface.  Roughly  speaking,  our  interpretation  of  surface 
contours  probably  embodies  the  following  geometric  reasoning  (the  vari¬ 
ous  deductions  follow  from  theorems  in  differential  geometry,  but  they 
will  not  be  elaborated  upon  here). 

1.  The  physical  curves  are  lines  of  curvature. 

2.  Intersecting  curves  meet  at  a  right  angle  on  the  surface  as  a 
consequence  of  (1). 

3.  Where  the  surface  contours  are  parallel,  the  surface  is  singly 
curved  (like  a  rippled  carpet). 

4.  Because  of  (1)  and  (3)  the  curves  are  geodesics  and  planar. 

The  above  provides  the  likely  basis  for  our  perception  of  shape  from 
surface  contours  in  images  such  as  figure  B-l.  Rigorous  psychophysical 
verification  of  these  conjectures  remains  to  be  performed,  but  informally 
our  visual  impression  of  shape  is  consistent  with  the  geometric  con¬ 
straints  1  through  4.  If  that  is  the  case,  those  constraints  may  also 
serve  as  the  basis  for  design  rules  relevant  to  simulators,  as  described 
by  the  following. 

When  modeling  terrain,  especially  undulating  terrain,  it  is  common¬ 
place  to  use  curves  that  resemble  the  edges  of  fields,  roads,  or  fences. 
These  curves  carry  information  about  the  terrain  shape  just  as  figure  B-l 
does,  but  only  if  they  are  restricted  geometrically  in  the  manner  that 
the  human  visual  system  implicitly  assumes.  That  is  to  say,  the  curves 
that  a  simulator  engineer  lays  in  the  terrain  model  should  have  the  fol¬ 
lowing  restrictions: 

1.  Each  curve  should  be  planar,  a  line  of  curvature,  and  geodesic. 

2.  Any  intersecting  curves  should  meet  at  a  right  angle. 

3.  If  the  surface  is  singly  curved,  like  a  smooth  ridge,  it  is 
only  necessary  to  depict  a  few  lines  of  greatest  curvature 
(as  in  figure  B-la) .  The  lines  parallel  to  the  ridge  may  be 
omitted  in  general. 

4.  If  the  surface  is  doubly  curved,  like  a  round  hill,  it  is  im¬ 
portant  to  place  lines  of  curvature  at  close  spacing.  It  is 
best  to  include  both  sets  of  lines  of  curvature,  so  the  sur¬ 
face  appears  to  have  a  net  draped  over  it  (figure  B-la).  Each 
intersection  will  then  be  a  right  angle,  and  the  surface  shape 
will  be  readily  apparent. 

These  are  a  few  rules  which  should  be  followed.  Note  that  they  may  be 
"broken"  and  we  might  still  correctly  perceive  the  surface  from  motion, 
just  as  we  can  understand  a  strange  piece  of  sculpture  by  walking  about 
it  and  seeing  it  from  different  perspectives.  For  instance,  suppose  a 
ridge  were  depicted  by  a  contour  that  climbs  over  it,  but  instead  of 
following  a  line  of  greatest  curvature  (the  "fall  line"  as  a  skier  would 
put  it)  it  climbs  obliquely  over  the  ridge.  At  a  glance  that  curve 
would  mislead  us — we  would  see  the  surface  shape  incorrectly,  but  with 
changing  viewpoints  we  might  later  see  the  surface  correctly  and  under¬ 
stand  that  the  curve  is  not  what  we  assumed. 

Depicting  surface  shape  by  undulating  lines,  as  just  discussed, 
should  be  compared  with  the  method  of  using  checkerboards.  First  of 
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all,  checkerboards  have  been  proposed  because  they  are  relatively  eco¬ 
nomical  in  terms  of  edges,  and  squares  seen  in  perspective  are  easily 
interpreted  in  3-D.  A  checkerboard  pattern,  in  fact,  is  probably  opti¬ 
mal  for  depicting  a  planar  surface,  given  the  current  limitations  in 
displayable  edges.  But  the  straightforward  extension  of  checkerboards 
to  indulating  surfaces  may  not  be  the  best  approach.  First  let  us  con¬ 
sider  singly  curved  surfaces — ripples  in  a  carpet.  Checkerboard  pat¬ 
terns  are  useful  for  ridges  so  long  as  the  rows  are  arranged  parallel 
with  the  ridge.  In  that  case  we  have  the  geometric  arrangement  of 
figure  B-lb  where  the  edges  of  the  squares  that  are  parallel  with  the 
ridge  (in  other  words,  those  that  lie  at  constant  elevation)  are  lines 
of  least  curvature,  and  the  other  edges  climb  over  the  ridge.  But  as 
stated  earlier,  the  lines  of  least  curvature  may  probably  be  omitted 
because  they  are  redundant.  In  terms  of  edges,  then,  a  series  of 
stripes,  rather  than  checkers,  would  be  more  economical  and  probably 
would  be  equally  effective.  For  a  doubly  curved  surface  such  as  a  round 
hill,  the  problem  becomes  somewhat  more  difficult,  but  it  is  solvable 
if  sufficiently  fine  stripes  or  checkers  are  used.  That  is  because 
a  doubly  curved  surface,  if  sliced  fine  enough,  can  be  treated  as  singly 
curved  in  general.  Incidentally,  the  line  defined  by  a  column  or  row  of 
checker  squares  has  alternating  contrast  along  its  length.  This  tends 
to  break  up  the  lines,  for  the  visual  system  has  difficulty  in  aggre¬ 
gating  into  a  line  elements  with  opposite  contrast  sense.  The  checker¬ 
board  pattern  is  unlikely  treated  as  a  collection  of  long  surface  con¬ 
tours,  therefore.  Instead,  each  white  or  black  square  is  seen  disjointly. 
For  that  reason,  long  stripes  or  lines  may  be  more  useful  in  depicting 
undulating  terrain. 


Sources  of  Information  About  Orientation 

The  previous  discussion  covered  the  major  sources  of  visual  infor¬ 
mation  about  surface  shape.  Next  we  will  discuss  orientation.  This 
discussion  is  more  difficult,  because  3-D  shape  is  most  naturally  de¬ 
scribed  relative  to  the  viewer  and  therefore  in  representing  shape  one 
simultaneously  captures  several  aspects  of  orientation.  For  instance, 
recall  that  in  the  night  landing  simulation,  motion  parallax  involving 
mere  luminous  dots  on  a  dark  background  can  give  a  compelling  impres¬ 
sion  of  movement  toward  the  solid  earth  during  landing.  Clearly  we  per¬ 
ceive  the  orientation  of  the  terrain  as  well  as  its  shape.  While  orien¬ 
tation  and  shape  are  intertwined,  it  is  useful  to  approach  visual 
processing  from  one  or  the  other  perspective,  depending  on  the  need. 

The  direction  of  movement  and  one’s  spatial  orientation  are  two 
forms  of  orientation  information  that  are  relatively  distinct  from 
shape.  Both  are  difficult  to  visually  determine  in  LLF  since  uneven 
terrain  often  prevents  one  from  using  a  distant  horizon  as  an  altitude 
reference.  But  optic  flow  seems  useful  (Gibson,  1950),  particularly 
in  LLF,  where  the  terrain  appears  smeared  and  blurred  (see  Marker  and 
Jones,  1980).  However,  the  highly  simplified  CIG  displays  might  not 
provide  sufficiently  dense  surface  texture  moving  with  sufficient 
smoothness  of  motion  to  be  effective.  Again,  we  see  that  further 
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research  is  needed  to  see  how  optic  flow  is  visually  processed.  This 
research  problem  is  of  particular  importance  to  LLF.  A  final  point 
regarding  optic  flow  should  be  made.  Simply  because  the  ground  tex¬ 
ture  appears  blurred  and  streaky  in  LLF  does  not  mean  that  one  can 
display  a  simple  pattern  of  streaks  in  the  periphery  displays.  When 
one  tracks  a  detail  on  the  ground,  it  becomes  roughly  stationary  on 
the  retina  and  the  smearing  of  detail  vanishes.  We  do  not  know  what 
role  tracking  eye  movements  would  play  in  LLF  (certainly  they  are  im¬ 
portant  when  scrutinizing  ground  targets,  but  what  importance  they  have 
in  the  basic  problem  of  flying  LLF  is  not  yet  known). 

A  local  "cue"  to  orientation  which  has  been  incorporated  in  CIG 
displays  is  the  simple  geometric  arrangement  of  two  lines  or  edges  that 
intersect  at  right  angles  on  the  surface  (figure  B-2a) .  It  is  straight¬ 
forward  to  show  that  the  image  of  a  right  angle  (which  because  of  fore¬ 
shortening  in  the  perspective  projection  appears  as  an  obtuse  angle) 
carries  some  information  about  the  surface  orientation,  but  it  alone 
does  not  specify  a  unique  3-D  orientation;  it  could  theoretically  cor¬ 
respond  to  a  large  variety  of  differently  oriented  planes  in  3-D.  Ad¬ 
ditional  constraint  comes  from  assuming  that  the  two  lines  are  equal 
length  in  3-D.^  Note  that  these  two  constraints — perpendicular  inter¬ 
sections  and  equal  length  segments — are  neatly  combined  in  the  square 
checkerboard  patterns  which  are  often  used  in  terrain  simulation. 
Perpendicular  intersections  in  general  are  useful  indicators  of  sur¬ 
face  orientation,  provided  the  lines  or  edges  are  of  equal  length  on 
the  surface. 


FIGURE  B-2  The  intersecting  lines  in  (a)  appear  to  lie  on  planar  sur¬ 
faces  in  3-D.  The  interpretation  involves  assuming  the  in¬ 
tersection  is  perpendicular  and  the  two  lines  are  of  equal 
length  on  the  surface.  The  intersections  in  (b)  may  be 
thought  of  as  a  signpost  at  a  street  intersection  seen  from 
different  viewpoints.  To  interpret  this  configuration,  it 
is  necessary  only  to  assume  the  intersecting  lines  are  mu¬ 
tually  perpendicular  in  3-D. 
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There  is  still  an  ambiguity  of  the  Necker  cube  variety.  But  when  view¬ 
ing  this  configuration  within  the  larger  context  of  the  ground  plane, 
that  ambiguity  is  resolved. 
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A  point  that  has  seldom  been  emphasized  is  that  a  trihedral  inter¬ 
section,  three  mutually  perpendicular  lines  (figure  B-2b) ,  actually  car¬ 
ries  enough  information  in  its  image  to  fix  its  surface  orientation 
precisely. I®  The  arrangement  resembles  a  signpost  at  a  street  inter¬ 
section — note  that  the  line  segments  no  longer  need  to  be  equal  length 
in  3-D.  This  configuration  would  be  a  very  effective  means  for  indi¬ 
cating  orientation,  even  scattered  across  an  otherwise  featureless 
terrain. 


Sources  of  Information  About  Scale 

Now  let  us  suppose  that  the  terrain  simulation  is  adequate  in  terms 
of  shape  and  orientation.  The  problem  that  would  remain  is  to  ensure 
that  the  pilot  can  judge  the  distance  to  the  surface  and  his  altitude 
above  it  with  sufficient  accuracy.  What  are  the  sources  of  information 
about  scale  that  he  might  utilize?  Scale  may  be  inferred  by  several 
perceptual  methods:  directly  from  stereopsis  or  indirectly  from  the 
known  size  of  recognizable  objects,  from  known  velocity,  and  several 
other  methods  that  we  will  discuss.  Scale  is  certainly  necessary  for 
various  flying  tasks;  the  difficulty  that  pilots  often  have  in  landing 
amphibious  planes  on  water  and  maintaining  low-level  flight  over  water 
and  sand  dunes  suggests  that  there  are  real  visual  scenes  that  are  de¬ 
ficient  in  information  about  scale. 

Stereopsis  Stereopsis  may,  in  theory,  have  a  minor  contribution 
at  low  altitude,  but  to  see  this  we  need  to  be  somewhat  quantitative 
for  a  moment.  Of  course,  stereopsis  is  most  useful  in  the  very  near 
environment  (distances  of  less  than  30  feet),  but  we  are  sensitive  to 
stereo  disparity  out  to  roughly  1,800  feet  (confuted  from  a  stereo¬ 
acuity  of  24  arc  seconds  (Graham  et  al. ,  1949)).  Hence  we  cannot  dismiss 
the  possibility  that  stereopsis  provides  a  pilot  altitude  information 
when  flying  LLF  (some  pilots  reportedly  look  out  of  the  side  canopy  to 
judge  their  altitude  (Harker  and  Jones,  1980)).  At  roughly  100  feet  in 
altitude,  one  can  see  downward  sufficiently  well  to  be  viewing  surface 
detail  within  the  useful  range  of  stereopsis  (Kennedy  and  McKechnie, 
1970),  but  because  of  the  blurring  and  streaking  which  we  have  dis¬ 
cussed,  unless  the  pilot  tracks  some  feature  on  the  ground,  stereopsis 
would  probably  fail.  But  a  significant  impression  of  depth  might  be 
derived,  provided  one  tracks  a  surface  feature  for  sufficient  time  to 
achieve  stereopsis  before  it  is  out  of  view.  Consider  an  aircraft 
traveling  500  knots  at  100  feet  and  a  surface  feature  that  passes 
within  500  feet  of  the  aircraft  at  nearest  approach.  One  may  easily 
compute  that  approximately  two  seconds  elapse  from  the  time  the  point 
enters  stereo  range  (1,800  feet)  to  when  it  passes  directly  by  the  side 
of  the  aircraft.  Hence,  if  a  pilot  in  low-level  flight  has  the  time 
to  visually  track  a  surface  feature  during  the  time  it  is  within  stereo 
range,  stereopsis  could  conceivably  provide  useful  scale  range.  It 
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Again,  up  to  a  Necker  reversal  which  can  be  disregarded  here. 
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must  be  stressed,  however,  that  head  movements  would  be  required  to 
track  the  surface  feature,  the  visual  tracking  task  would  not  be  easy 
in  a  hostile  environment,  and  stereopsis  under  those  visual  conditions 
has  not  received  sufficient  psychophysical  study. 

There  is  evidence  that  casts  doubt  on  this  hypothesis,  however. 

The  evidence  is  simple:  recall  that  certain  types  of  terrain  are  par¬ 
ticularly  difficult  to  fly  over  at  low  level — specifically,  sand  dunes 
and  open  water.  There  is  sufficient  viewing  time  to  achieve  binocular 
fusion,  sufficient  visible  detail  on  which  to  establish  stereo  fusion, 
and  a  large  region  of  the  surface  is  within  stereo  range  at  'low  alti¬ 
tude.  Nonetheless,  LLF  is  difficult  over  that  terrain.  Apparently 
the  sense  of  scale  (if  any)  that  one  derives  from  stereopsis  is  insuf¬ 
ficient  for  making  the  critical  altitude  judgments.  Stated  another 
way,  stereopsis  may  play  some  role  in  LLF,  but  it  is  probably  insig¬ 
nificant.  It  must  be  said,  however,  that  the  above  is  not  conclusive-- 
the  issue  deserves  careful  experimental  consideration. 

Known  Size  One  method  to  infer  distance  is  from  the  measured 
retinal  size  of  an  object  of  known  size.  It  is  well  known  that  the 
distance  to  an  object  varies  inversely  with  the  angle  it  subtends. 
Theoretically,  if  one  knew  that  a  tree  were  20  feet  high,  one  could 
also  know  its  absolute  distance  on  the  basis  of  its  retinal  size. 
Certainly  the  visual  system  capitalizes  on  this  relation,  and  we  even 
consciously  search  a  novel  scene  for  some  cue  to  its  scale  (conscious 
attention  is  not  required,  as  Helmholtz  (1925)  recognized  in  his  term¬ 
ing  it  "unconscious  inference") .  There  are  phenomena  which  strongly 
suggest  that  we  do  infer  distance  frou  known  size  (e.g.,  Enright,  1970). 

However,  empirical  studies  of  "size  constancy"  and  absolute  dis¬ 
tance  perception  (Epstein  and  Landauer,  1969;  Gogel,  1971;  Hochberg, 
1971;  Rock  and  McDermott,  1964)  have  shown  that  the  psychophysical  re¬ 
lation  between  distance  and  retinal  size  is  not  as  simple  as  might  be 
expected.  This  is  one  place  where  simulator  designers  have  experienced 
difficulty  in  interpreting  the  psychological  literature.  Many  of  the 
experiments  were  performed  in  artificially  restricted  viewing  situations 
(e.g.,  darkened  rooms  with  few  reference  objects)  and  those  that  were 
performed  "in  the  field"  would  involve  verbal  judgments  of  distance, 
e.g.,  as  a  target  being  so  many  feet  away  (Gibson  and  Bergman,  1954) 
or  comparison  judgments  between  two  distances  (Foley,  1972).  Few  ex¬ 
periments  reveal  just  how  precisely  we  perceive  the  scale  of  the  visual 
world  from  objects  of  known  size. 

It  must  be  stressed  that  we  are  ultimately  concerned  with  the 
visual  judgments  of  absolute  distances,  and  therefore  with  providing 
sufficient  information  so  that  the  pilot  can  fly  100  feet  above  the 
terrain,  for  instance.  The  known-size  method  probably  plays  a  role  in 
this  ability.  This  is  another  place  where  tightly  directed  investiga¬ 
tion  is  critically  needed. 

Known  Velocity  A  potentially  useful  method  for  computing  scale  is 
quite  similar  to  the  known-size  method  just  discussed.  If  one  knows 
the  velocity  of  travel  past  an  object,  one  can  infer  the  distance  to 
that  object  from  the  induced  angular  velocity  (e.g.,  retinal  velocity 
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given  a  fixed  gaze,  or  velocity  relative  to  some  point  on  the  cockpit 
canopy).  The  following  relation  between  absolute  distance  d_,  absolute 
velocity  V_,  and  angular  velocity  u  was  pointed  out  by  Gibson  and  others 
(Gibson,  1950;  Nakayama  and  Loomis,  1974): 

Vs  in  8 

d  =  ; -  (1) 


where  8  is  the  angle  from  the  direction  of  travel  to  the  given  point 
whose  distance  is  to  be  measured.  The  relation  presumes  that  the  given 
point  is  stationary  and  the  viewer  is  in  pure  translation.  It  is 
powerful  in  that  it  allows  one  to  compute  a  depth  map  for  the  entire 
visual  field  (everywhere  there  is  detail  moving  across  the  field  of 
view)  in  terms  of  w  and  6  (measurable,  in  theory)  and  the  single  un¬ 
known  V.  That  depth  map  would  specify  absolute  distance  if  tie  abso¬ 
lute  velocity  V  were  known. 

Conscious  deduction  is  not  necessary  or  even  likely  in  this  pro¬ 
cess.  Instead,  a  pilot  experienced  in  flight  at  low  altitude  and  high 
velocity  might  come  to  expect  a  particular  angular  velocity  in  the 
periphery,  in  a  manner  analogous  to  driving  a  car  and  expecting  the 
road  to  slip  by  at  an  appropriate  rate  for  any  given  driving  speed.!1 
Then  if  the  groundspeed  is  held  constant  but  the  altitude  is  lower  than 
normal,  the  angular  velocity  would  be  higher  than  normal,  and  vice  versa. 
Tv  is  relation  could,  theoretically,  account  for  the  precision  with 
which  altitude  may  be  held  in  LLF  by  experienced  pilots.  It  is  not 
inconceivable  that  part  of  the  skill  that  a  pilot  acquires  through 
flight  training  is  the  unconscious  calibration  of  retinal  velocity  ac¬ 
cording  to  ground  speed  and  altitude,  and  furthermore  the  development 
of  an  effective  feedback  loop  that  attempts  to  control  altitude  accord¬ 
ing  to  actual  retinal  velocity  at  any  instant. 

It  should  be  relatively  straightforward  to  establish  whether  this 
is  the  case.  The  direct  relationship  between  altitude  and  groundspeed 
predicts  that  as  speed  increases  there  should  be  a  tendency  to  increase 
the  cruise  altitude,  other  factors  being  constant.  The  following  evi¬ 
dence  (J.  Richter,  personal  communication)  suggests  that  this  occurs. 
Interestingly,  it  is  most  apparent  when  flying  over  open  water  (the 
situation  we  noted  before  as  difficult).  Perhaps  the  fact  that  water 
provides  little  evidence  of  scale,  compared  to  a  richly  textured  rural 
terrain  containing  familiar  surface  features,  allows  one  to  observe 
the  weaker  contribution  of  the  known  speed  method.  Over  the  ground, 
one's  altitude  tends  to  remain  fairly  constant  as  speed  increases,  but 
over  water  the  altitude  definitely  tends  to  climb  as  the  aircraft  is 
accelerated.  It  is  consistent  with  there  being  some  unconscious 


It  is  noteworthy  that  wnen  changing  from  a  passenger  car  to  a  low 
sports  car  the  apparent  speed  is  greater  merely  because  of  the  larger 
angular  velocities  associated  with  being  nearer  tne  ground. 
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process  attempting  to  maintain  an  expected  angular  velocity,  where  an 
increase  in  angular  velocity  is  attributed  to  a  loss  of  altitude  ra¬ 
ther  than  an  increase  in  ground  speed.  These  informal  reports  suggest 
that  the  known  velocity  method  for  computing  scale  may  have  some 
validity  in  actual  flight,  and  therefore  should  be  given  careful  ex¬ 
perimental  investigation.  The  experiments  should  probably  be  performed 
using  unfamiliar  surface  textures  to  examine  the  relationship  between 
speed  and  altitude.  The  problem  must  be  addressed  in  conjunction  with 
an  investigation  of  texture,  as  undoubtedly  the  effect  will  vary  in 
magnitude  with  texture  variations. 

Other  Methods  In  addition  to  the  two  methods  just  discussed, 
there  are  other  geometrical  methods  for  gaining  scale  information.  ' 
Several  have  been  discussed  by  Harker  and  Jones  (1980).  Typically  they 
consist  of  some  geometric  quantity — e.g. ,  the  ratio  of  angle  subtended 
by  some  vertical  surface  feature  compared  to  its  angular  distance  be¬ 
low  the  horizon — from  which  one  may  judge  whether  the  altitude  is  main¬ 
tained  constant,  or  whether  the  aircraft  will  clear  a  vertical  obsta¬ 
cle.  while  strictly  speaking  these  geometric  quantities  do  not 
specify  scale  in  the  sense  we  mean,  they  are,  nonetheless,  potentially 
useful  in  flying.  We  probably  should  distinguish  between  qualitative 
problems  that  can  be  solved  geometrically  (such  as  obstacle  clearance 
and  maintaining  constant  altitude)  and  quantitative  problems  (such  as 
clearing  obstacles  by  so  many  feet,  maintaining  a  particular  altitude, 
and  so  forth).  But  it  is  not  certain  which  sort  of  problem,  the  quali¬ 
tative  or  the  quantitative,  deserves  the  greater  attention  at  the  moment. 


Sketch  of  an  Application  of  the  Computational  Approach 

To  illustrate  a  specific  example  of  using  the  computational  ap¬ 
proach  to  improving  the  CIG  simulation  of  terrain,  consider  the  depic¬ 
tion  of  undulating  terrain  for  LLF.  We  focus  on  the  problem  of  improv¬ 
ing  the  visuai  information  about  the  shape  of  the  terrain,  which  of 
cour?  is  but  one  aspect  of  the  overall  effort  of  generating  an  effec¬ 
tive  LLF  visual  simulation. 

Reflecting  on  the  various  sources  of  visual  information  about 
shape,  we  recall  that  these  include  visual  motion  (of  surface  features 
due  to  one’s  movement  over  the  terrain),  shading,  texture  gradients, 
and  surface  contours.  Suppose  we  focus  more  closely  on  the  contribu- 
tuion  that  surface  contours  make  to  the  apparent  shape  of  the  terrain. 
That  is,  let  us  consider  how  one  might  make  terrain  features  such  as 
ridges  or  valleys  visually  apparent  by  essentially  painting  contours 
(perhaps  depicting  property  boundaries  or  roads)  across  the  CIG  terrain 
model. 


Some  are  well  known  and  supposed 
instance,  if  the  top  of  a  tower  is 
above  the  flight  path  and  will  not 


to  be  consciously  attended  to.  For 
climbing  in  the  visual  field  it  is 
be  cleared. 
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The  results  of  theoretic  analysis  of  surface  contour  perception 
were  sketched  earlier.  Basically,  our  3-D  interpretation  of  contours 
embodies  particular  assumptions  about  the  geometry  of  these  contours 
and  their  relation  to  the  surface  across  which  they  lie.  The  assump¬ 
tions  include  (see  earlier  and  Stevens,  1981): 

1.  The  physical  curves  are  lines  of  curvature. 

2.  Intersecting  curves  meet  at  a  right  angle. 

3.  Parallel  contours  imply  the  surface  is  singly  curved. 

4.  The  physical  curves  are  geodesic  and  planar. 

If  those  geometric  assumptions  are  indeed  those  governing  human  percep¬ 
tion  of  contours,  that  suggests  a  means  for  placing  contours  across  CIG 
terrain  models  so  that  they  are  visually  effective  (recall  the  four  "de¬ 
sign  rules"  for  surface  contours  described  earlier). 

The  next  step,  therefore,  is  careful  psychophysical  study  that 
verifies  that  surface  contour  perception  involves  the  geometric  assump¬ 
tions  that  were  established  on  purely  theoretic  grounds.  Specifically, 
consider  item  4  above.  If  our  perceptual  apparatus  incorporates  the 
assumption  that  a  viewed  curve  is  planar  and  geodesic  (in  the  absence 
of  information  to  the  contrary  as  provided  by,  say,  stereopsis  and  mo¬ 
tion),  that  assumption  has  predictable  and  testable  consequences.  What 
would  be  required  next,  therefore,  is  empirical  investigation.  Once  we 
have  established  the  role  of  ;  ese  (or  other  )geometric  assumptions,  a 
quantitative  study  is  required  of  the  relative  importance  of  such  factors 
as  contour  spacing,  line  width,  and  so  forth. 


Concluding  Remarks 

Three  types  of  3-D  information  that  must  be  gathered  by  the  visual 
system  in  order  that  a  pilot  might  fly  close  to  the  terrain  are  its 
scale,  shape,  and  orientation.  The  visual  deficiencies  of  the  LLF  simu¬ 
lation  can  be  discussed  in  such  terms.  Indications  are  that  scale  in¬ 
formation  requires  the  greatest  improvement. 

More  rigorous  analysis  comes  next.  We  gain  insight  into  improving 
the  display  by  learning  how  the  human  visual  system  determines  scale, 
for  example.  There  are  many  sources  of  scale  information,  but  much 
more  effort  is  required  before  we  can  make  concrete  suggestions  regard¬ 
ing  CIG  improvements. 

We  did  see  an  example  of  where  the  computational  analysis  of  sur¬ 
face  contours,  a  source  of  shape  information,  leads  to  rather  specific 
suggestions  regarding  the  visual  display.  For  our  visual  systems  to 
make  sense  of  image  curves  in  terms  of  actual  contours  lying  across 
physical  surfaces,  a  number  of  geometric  assumptions  have  to  be  made. 
Analyzing  what  these  might  be,  in  theory,  coupled  with  psychophysical 
verification  that  those  particular  assumptions  are  involved  in  our  in¬ 
terpretation  process  and  not  some  other  set  of  assumptions,  leads  to 
some  design  rules,  as  it  were.  The  design  rules  are  intended  for  the 
simulator  engineer  who  depicts  terrain  using  curves  (one  curve  might  be 
meant  to  depict  a  fence  over  a  hill,  another  might  depict  the  edge  of 
a  field).  It  is  of  course  important  that  the  pilot  perceive  the  simu¬ 
lated  world  in  the  way  that  it  was  intended  by  the  designer.  To  do  so. 
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t he  contours  placed  in  the  terrain  model  should  be  restricted  in  their 
geometry  so  that  they  match  the  geometric  assumptions  that  govern  their 
visual  interpretation. 

The  surface  contour  example  is  a  particularly  clean  demonstration 
of  how  understanding  some  aspect  of  vision  has  application  to  simula¬ 
tion.  As  should  be  evident  from  the  previous  discussions,  much  needs 
to  be  learned  about  vision. 
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APPENDIX  C 

VISUAL  SENSORY  ASPECTS  OF  SIMULATION 
One  Example  of  an  Experimental  Approach^ 

D.  M.  Regan 

Rationale 

Several  quite  different  strategies  have  been  suggested  for  coming 
up  with  visual  factors  that  might  correlate  with  flying  performance  in 
a  simulator  or  aircraft.  These  strategies  range  from  multivariable 
factorial  studies  to  the  "natural  computation"  approach  of  the  MIT 
group.  This  appendix  outlines  one  candidate  strategy  for  identifying 
visual  sensory  factors  that  are  important  in  flight  simulation.  This 
approach  runs  along  the  following  lines: 2 

1.  Carry  out  basic  research  on  the  sensory  aspects  of  vision, 
testing  some  specific  hypotheses  about  how  the  eye  processes 
visual  sensory  information. 

2.  From  the  results  of  this  basic  research,  predict  which  visual 
functions  will  be  important  in  simulation. 

3.  Test  these  visual  functions  in  pilots,  and  correlate  the 

test  results  with  simulator  (and  aircraft)  flying  performance. 

In  order  to  focus  our  efforts,  we  have  restricted  ourselves  to  one 
of  several  possible  hypotheses  as  a  guide.  The  next  section  of  this 
paper  lists  some  of  the  resulting  basic  research  findings  and  conclu¬ 
sions  that  might  be  relevant  for  simulation.  The  last  section  outlines 
two  follow-up  studies  of  simulator  flying  performance. 

We  should  note  that  the  theoretical  approach  used  in  these  simu¬ 
lator  studies  is  no  more  than  a  working  hypothesis  or  guide. 


It  is  often  noted  that,  in  general,  an  increased  basic  understanding  of 
visual  function  would  help  to  improve  simulator  design.  This  appendix 
can  be  read  as  an  account  of  one  (among  many)  practical  ongoing  efforts 
along  these  lines,  in  which  the  same  experimental  thrust  has  been  main¬ 
tained  over  a  ten-year  period.  The  simulator  studies  outlined  here 
could  by  no  means  have  been  carried  through  without  generous  help  and 
advice  from  staff  of  the  U.S.  Air  Force  Human  Resources  Laboratory, 
Williams  AFB  and  all  flying  personnel  involved  for  Williams  AFB,  Nellis 
AFB,  Luke  AFB,  and  MacDill  AFB. 

2 

We  appreciate  that  other,  quite  different,  strategies  have  been  used  in 
the  bulk  of  visual  research  on  simulation. 
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Our  theoretical  approach  has  been  firmly  in  the  camp  of  those  who 
use  simple  laboratory  stimuli,  often  at  threshold  levels,  with  the  aim 
of  uncovering  elementary  functional  subunits  in  the  visual  pathway 
(Braddick  et  al. ,  1978;  Regan,  1982),  If  we  wish  to  usefully  predict 
vision  in  a  complex  visual  environment,  we  should  search  for  subunits 
that  analyze  retinal  image  information  into  more  or  less  orthogonal 
(nonoverlapping)  elements.  However,  it  is  not  necessary  that  a  given 
subunit  be  linear. 


Examples  of  Basic  Research  Findings  and  Conclusions  with 
Possible  Implications  for  Simulator  Design 

The  optical  flow  pattern  generated  by  an  observer's  movement  con¬ 
tains  information  about  the  direction  of  movement  (Gibson,  1950,  1958; 
Richards,  1975).  It  has  been  sometimes  suggested  that  an  observer  could 
determine  his  projected  impact  point  from  the  center  of  expansion  of 
his  retinal  flow  pattern;  however,  there  is  reason  to  think  that  in 
practice  this  cannot  be  done  (Llewellyn,  1971;  Regan  and  Beverley, 

1981).  If  the  observer  fixates  his  gaze  off  to  one  side  away  from  his 
direction  of  movement,  the  retinal  flow  pattern  will  be  distorted 
(Regan  and  Beverley,  1981;  Richards,  1975).  On  the  other  hand,  subjects 
can  accurately  judge  where,  on  the  retinal  image,  the  local  magnifica¬ 
tion  is  changing  most  rapidly,  and  in  some  situations  this  direction 
coincides  with  the  direction  of  self-motion  (Regan  and  Beverley,  1981). 
There  is  evidence  that  visually  judging  the  direction  of  self-notion  on 
the  basis  of  visual  flow  patterns  may  involve  selective  sensitivity  to 
changing  size  (Regan  and  Beverley,  1979a). 

In  order  to  explain  certain  experimental  data,  we  theoretically 
proposed  that  the  human  eye  responds  to  an  object's  rate  of  change  of 
size  more  or  less  independently  of  the  object’s  motion  in  the  frontal 
plane  (Regan  and  Beverley,  1978,  1980).  Note  that  an  object's  rate  of 
change  of  size  is  closely  related  to  its  line-of-sight  component  of 
velocity.  In  particular,  the  response  of  the  eye's  changing-size  sub¬ 
system  may  directly  indicate  time  to  collision,  more  or  less  indepen¬ 
dently  of  the  observer's  trajectory  and  of  the  object's  trajectory. 3 

The  human  eye  processes  stereo  position  in  depth  separately  from 
stereo  motion  in  depth;  in  addition  to  the  classical  stereoscopic  depth 
mechanism  sensitive  to  disparity,  there  is  a  second  stereoscopic  mecha¬ 
nism  sensitive  to  the  relative  speed  seen  by  the  left  and  right  eyes. 
Physiologically,  there  are  different  types  of  brain  cells  predominantlv 
sensitive  to  disparity  and  stereo  motion  in  depth  (Regan  et  al . ,  1979; 
Richards  and  Regan,  1973). 

A  sensation  of  motion  in  depth  can  be  produced  either  by  stereo¬ 
scopic  (binocular)  stimulation  or  by  changing  size.  Relative  visual 
sensitivity  to  the  binocular  and  to  the  monocular  cues  does  not  much 
depend  on  viewing  distance  (contrast  with  classical  stereo  vision). 


3 

That  (Time  to  Collision)  =  (Object's  Fractional  Rate  of  Change  of 
Size)~3  is  easily  shown  for  a  nonrotating  object. 


Relative  binocular  to  monocular  sensitivity  does,  however,  depend  on 
the  object's  speed  and  linear  size.  This  suggests  that  there  is  no 
general  answer  to  the  question,  "What  effect  has  closing  one  eye  upon 
a  pilot’s  ability  to  judge  motion  in  depth?"  (Regan  and  Beverley, 
1979b). 

Findings  suggest  that  for  motion-in-depth  perception,  static 
texture  is  worse  than  no  texture  (Beverley  and  Regan,  submitted). 


Correlations  Between  Visual  Test  Results  and 
Simulator  Flying  Performance 

This  section  describes  two  studies  in  which  the  results  of  visual 
tests  on  pilots  were  compared  with  flying  performance  on  a  flight  simu¬ 
lator  (A-10  aircraft)  and  with  aircraft  flying  grades  (T-38  jet  train¬ 
er)  at  Williams  AFB. 

In  the  first  study  (Kruk  et  al.,  1981),  subject  groups  comprised 
twelve  pilot  instructors,  twelve  nonpilot  weapons  officers,  twelve 
graduating  student  pilots,  twelve  first-year  student  pilots,  and  twelve 
nonmilitary  nonflying  subjects.  A  total  of  thirteen  visual  test  mea¬ 
sures  were  collected  from  each  subject,  including  manual  tracking  of  a 
target  that  moved  in  depth  in  the  presence  of  vibratory  "jitter"  motion 
(Beverley  and  Regan,  1980).  The  simulator  flying  task  was  restricted 
visibility  landing.  In  brief,  the  better  the  pilot’s  performance  in 
tracking  a  target  that  moved  in  the  frontal  plane,  the  earlier  was  the 
first  visual  flight  correction  during  landing.  In  addition,  the  number 
of  crashes  was  predicted  by  tracking  accuracy  for  a  target  that  moved 
in  depth.  Since  the  only  difference  between  the  two  tracking  tests 
was  visual,  motor  factors  being  common,  and  since  the  two  tests  corre¬ 
lated  with  different  aspects  of  landing  performance,  we  concluded  that 
the  two  tracking  tests  detected  intersubject  differences  in  visual  sen¬ 
sory  factors  that  are  important  in  landing  (though  intersubject  dif¬ 
ferences  in  eye-hand  coordination  may  well  have  been  involved  also) . 
Quite  different  correlations  were  obtained  between  landing  performance 
and  psychophysical  thresholds  for  motion  and  contrast.  Since  these 
correlations  were  weak,  we  concluded  that  these  threshold  tests  failed 
to  detect  the  performance-related  intersubject  differences  in  vision 
detected  by  the  tracking  tests. 

Different  individual  subjects  were  used  in  a  second  study.  Simu¬ 
lator  flying  performance  was  compared  with  the  results  of  sensory  visual 
tests  for  twelve  experienced  fighter  pilots  (group  1),  twelve  pilot 
instructors  (group  2),  and  twelve  graduating  student  pilots  (group  3). 
The  following  is  extracted  from  a  report  submitted  for  publication 
(Kruk  et  al.,  submitted  for  publication). 

Flight  Simulator  Tasks 

The  A-10  cockpit  of  the  Advanced  Simulator  for  Pilot  Train¬ 
ing  (ASPT)  at  Williams  AFB  was  used  throughout.  The  simulator 
was  initialized  146  m  behind  an  A-10  lead  aircraft  and  the  pilot 
instructed  to  close  and  assume  fingertip  right  position  until 


i 


9 


68 


the  simulator  re-initialized  after  2  min  flight.  This  pro¬ 
cedure  was  repeated  for  close  trail  and  fingertip  left  posi¬ 
tions,  then  the  whole  sequence  was  repeated  with  the  horizon 
removed.  Time  spent  in  correct  formation  was  measured. 

Groups  1  and  2  then  flew  a  low-level  exercise.  Pilots  were 
instructed  to  fly  through  a  mountain  pass  and  on  for  about 
5000  m  while  maintaining  an  altitude  of  about  36  m,  then  pop 
up  to  about  360  m  and  execute  a  dive  bomb  attack  on  a  speci¬ 
fied  target  (one  of  five  buildings  near  a  road).  Then  the 
pilot  flew  about  3000  m  at  about  36  m  altitude  to  exit  the 
area.  There  were  two  SAM  sites  and  two  active  antiaircraft 
artillery  sites.  Auditory  tones  indicated  to  pilots  when 
they  were  being  tracked  by  SAM  radar,  when  a  missile  was 
launched  and  when  a  missile  was  close,  respectively.  Pilots 
were  instructed  that  they  were  likely  to  be  shot  down  if  they 
flew  above  about  110  m  for  more  than  about  10  sec.  Each 
pilot  carried  out  this  exercise  six  times.  The  SAM  and  AAA 
threats  served  to  ensure  that  performance  would  be  strongly 
influenced  by  visual  factors,  since  low-level  flight  is 
heavily  dependent  on  out-of-cockpit  visual  input.  Moreover, 
since  the  target  could  not  be  seen  from  low  altitude,  visual 
judgments  were  important  in  judging  when  to  pop  up  and  in 
correctly  placing  the  aircraft  during  the  dive.  Bomb  aiming 
and  release  were  entirely  visual  and  under  the  pilot’s  con¬ 
trol.  The  manual  bomb  delivery  mode  was  employed  so  that 
accuracy  was  entirely  determined  by  the  aircraft's  trajectory 
and  speed  at  the  moment  of  release.  We  measured  the  percent 
time  tracked  by  missile  radar,  number  of  tilths  shot  down, 
number  of  crashes,  number  of  bomb  delivery  hits  and  near 
misses  (within  36  m  radius),  altitude  and  heading  variations, 
variability  in  release  height,  and  release  "G." 

Restricted  visibility  landings  were  carried  out  as  de¬ 
scribed  [in  Kruk  et  al.,  1981].  In  brief,  pilots  were  told 
that  airspeed  was  120  kts  (62  m/s),  glide  slope  2.5°,  and 
that  the  initial  ILS  (Instrument  Landing  System)  flight  path 
was  randomly  set  to  land  61  m  to  left  or  right  of  centre. 
Visibility  was  nominally  460  m  so  that  the  runway  was  in¬ 
visible  early  in  the  approach.  Pilots  were  instructed  to 
land  on  each  approach,  and  each  pilot  completed  six  landings. 
We  measured  the  number  of  crashes,  and  the  distance  from  run¬ 
way  threshold  at  which  the  first  visual  flight  correction  was 
made. 

Visual  tests  comprised  superthreshold  velocity  discrimi¬ 
nation  of  a  radially  expanding  flow  pattern,  manual  tracking 
of  both  motion  in  depth  and  motion  in  the  frontal  plane,  mo¬ 
tion  thresholds  and  contrast  thresholds  for  a  moving  square, 
and  a  static  sinewave  grating. 

Landing  and  formation  flight  performance  correlated  with 
both  manual  tracking  and  expanding  flow  pattern  test  results. 
Pilots  who  were  better  able  to  discriminate  different  rates 
of  expansion  of  the  test  flow  pattern  achieved  a  greater 
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percentage  of  hits  and  near  misses  in  the  low-level  flight 
and  bombing  task.  Aircraft  flying  grades  for  student  pilots 
correlated  with  expanding  flow  pattern  test  results  and  with 
manual  tracking  of  motion  in  depth. 

Implications  for  Simulator  Design  and  Pilot  Selection  Tests 

Of  the  13  threshold  and  superthreshold  test  measures, 
tests  of  superthreshold  visual  sensitivity  to  motion  corre¬ 
lated  most  strongly  with  simulator  and  aircraft  flying  per¬ 
formance.  This  indicates  that  intersubject  differences  in 
the  physiological  processing  of  motion  were  an  important  de¬ 
terminant  of  intersubject  differences  in  flying  performance 
on  our  particular  tests.  Our  findings  suggest  that  motion 
tests  (e.g.,  the  changing-size  tracking  test  and  superthresh¬ 
old  velocity  discrimination)  might,  perhaps,  be  usefully 
added  to  the  present,  mainly  static,  battery  of  visual  tests 
used  in  selecting  flying  personnel.  Our  findings  also  sug¬ 
gest  that  the  transfer  of  training  from  simulator  to  aircraft 
flying  tasks,  including  low-level  flight,  restricted  visibil¬ 
ity  landing  and  formation  flight,  might  be  enhanced  by  placing 
more  emphasis  on  improving  the  accurate,  artifact-free  repre¬ 
sentation  of  motion  in  simulator  visual  displays  as  compared 
with  the  historical  emphasis  on  improving  static  spatial  fac¬ 
tors  such  as  resolution  and  number  of  edges. 

Ongoing  Research 

In  cooperation  with  Yuma  Range  ar.d  Williams  AFB  we  are 
currently  working  on  extending  the  simulator  studies  to  in¬ 
clude  performance  measures  in  aircraft,  with  the  aim  of  find¬ 
ing  whether  the  visual  factors  we  have  identified  as  important 
in  certain  simulator  flight  tasks  are  also  important  in  the 
air. 


Some  Suggested  Experiments 

1.  Define  the  role  and  important  parameters  of  expanding  flow  pat¬ 
terns  in  judging  the  direction  of  self-motion  (e.g.,  spatial  frequency 
and  dynamic  requirements). 

2.  Find  whether  visual  sensitivity  to  stereo  motion-in-depth  is 
involved  in  low-level  flight  in  fixed-wing  aircraft  and  helicopters. 

3.  Define  the  role  of  visual  sensitivity  to  changing  size  in 
low-level  flight. 

4.  Define  the  role  of  visual  sensitivity  to  texture  shear  in 
recognizing  objects  and  judging  distance  in  flying  environments,  in¬ 
cluding  nap-of-the-earth  operations. 
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APPENDIX  D 

LOCOMOTION  THROUGH  VISUAL  SPACE:  OPTIC  FLOW  AND 
AND  PERIPHERAL  VISION 

Ralph  Norman  Haber 


The  design  of  flight  simulation  displays  for  low-level  flight  must 
necessarily  compromise  a  realistic  presentation  of  the  full  visual 
scene  unfolding  before  and  beneath  the  pilot.  It  is  simply  not  possi¬ 
ble  to  provide  even  reasonable  verisimilitude  of  detail  over  the  entire 
visual  field  of  the  pilot  at  rates  approaching  the  real  time  demands  of 
flight.  The  faster  the  velocity  and  the  lower  the  altitude,  the  more 
severe  the  compromise.  And  the  compromise  is  extreme  for  low-velocity 
flight  as  well,  though  not  because  of  real-time  limitations.  Of  course, 
low-level  flight  simulation  has  been  practical  for  a  number  of  years, 
with  the  exercise  of  many  compromises  in  how  visual  information  is  pre¬ 
sented  to  the  pilot.  Most  of  these  compromises  have  resulted  from  hard¬ 
ware  limitations,  and  not  necessarily  including  an  analysis  of  how  the 
human  visual  system  processes  information  about  locomotion  through  space. 

In  order  to  examine  the  impact  of  these  compromises,  especially  in 
a  context  of  the  information  human  beings  use  to  perceive  visual  space, 
the  following  is  a  brief  visual  task  analysis  that  relates  sources  of 
visual  information  available  in  different  flight  tasks,  especially 
those  that  can  or  do  benefit  from  training  in  flight  simulators. 


Visual  Flight  Task  Analysis 

Sources  of  visual  information  available  to  a  perceiver  when  view¬ 
ing  a  natural  scene  while  moving  through  it  can  be  grouped  into  three 
general  and  independent  classes:  those  contained  in  the  pattern  of 
disparities  that  exist  between  the  two  retinal  images;  those  contained 
in  the  patterning  of  the  continuous  relative  displacements  of  projected 
light  on  each  retina  that  occur  as  the  plane  moves;  and  those  contained 
in  the  perspective  transformation  of  the  light  reflected  from  surface 
features  as  those  features  are  projected  on  each  retina.  (1  am  exclud¬ 
ing  and  ignoring  several  potential  local  sources  of  information,  prob¬ 
ably  of  little  importance  when  looking  at  natural  scenes,  but  perhaps 
used  in  the  laboratory  or  other  impoverished  circumstances.  Examples 
are  accommodative  and  convergence  corrections  of  the  eyes  as  voluntary 
fixations  shift  from  near  to  far  or  vice  versa.)  Each  of  these  three 
classes  provides  information  about  the  layout  of  space  information 
that  can  fully  specify  the  arrangements  of  space,  such  as  surface 
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orientation  and  scale,  observer  position  and  change  in  position  rela¬ 
tive  to  the  surface,  and  location  of  all  objects  attached  to  the  ground 
surface . 

But  the  specification  of  the  layout  of  space  is  quite  different 
for  each  class  of  sources,  with  substantial  differences  in  how  the  in¬ 
formation  is  processed.  For  example,  information  produced  by  observer 
motion  is  available  only  when  the  observer  in  fact  moves  fast  enough 
relative  to  his  distance  to  the  terrain  surface  so  that  the  differences 
in  displacement  of  edges  on  the  retina  are  above  threshold.  There  is 
little  motion  perspective  information  available  to  a  hovering  helicop¬ 
ter  pilot,  regardless  of  the  nonuniformity  of  the  terrain,  nor  to  a 
bomber  pilot  at  50,000  feet  even  at  supersonic  speeds  when  the  terrain 
is  relatively  flat.  Further,  binocular  disparity  information  is  avail¬ 
able  only  when  the  observer's  point  of  regard  is  close  enough  to  the 
surfaces  being  looked  at  so  as  to  be  above  stereoacuity  thresholds. 
Since  the  two  retinal  images  produced  by  a  scene  are  virtually  identi¬ 
cal  when  the  scene  is  more  than  several  hundred  meters  distant,  insuf¬ 
ficient  disparity  is  present  to  be  detectable  and  processed.  Hence, 
in  theory,  for  most  flying  tasks,  adding  a  second  eye  does  not  add  much 
information  beyond  the  increased  field  of  view.  This  conclusion  does 
not  hold  in  theory  for  most  helicopter  flying  tasks  or  for  air-to-air 
refueling,  for  example,  since  for  those  tasks  visual  distances  are  well 
within  stereoacuity  limits  so  that  disparities  are  above  threshold. 

In  practice,  some  low-level  flight  tasks  even  up  to  several  hundred 
meters  may  benefit  from  binocular  disparity  information,  but  this  is 
yet  to  be  demonstrated. 

While  motion-produced  visual  information  requires  a  moving  ob¬ 
server  and  stereopsis  requires  the  surfaces  of  the  scene  to  be  nearby, 
information  from  the  perspective  transformation  of  surface  features  is 
theoretically  always  available,  even  to  a  one-eye  stationary  observer. 
All  that  is  needed  are  perceptible  surface  features  such  as  textured 
surfaces  and  objects;  contrasts  in  luminance  or  color  between  adjacent 
objects,  between  objects  and  the  ground,  or  between  different  facets 
of  each  object;  and  of  course  contrasts  produced  by  shadows  or  varia¬ 
tion  in  surface  orientation  relative  to  the  sources  of  illumination. 

The  grain  of  these  tea tures  differs,  so  at  near  distance  the  texture 
of  the  ground  substance  itself  may  be  above  threshold,  whereas  the 
same  scene  viewed  from  a  great  distance  has  a  texture  composed  of  the 
spacing  of  ground  objects,  such  as  trees  or  hills. 

The  above  discussion  of  the  kinds  of  visual  information  available 
from  natural  scenes  suggests  important  restrictions  when  considering 
flight  simulation  for  various  flying  tasks.  For  example,  a  stereo¬ 
scopic  display  is  probably  important  for  helicopter  pilot  training, 
but  is  not  likely  to  provide  any  useful  visual  information  for  any 
fixed-wing  aircraft  operation  tasks  (except  perhaps  air-to-air  refuel¬ 
ing)  because  all  such  tasks  place  the  pilot  at  too  great  a  distance 
from  other  visible  surfaces  or  objects.  Some  careful  work  is  needed 
to  justify  this  conclusion,  since  it  may  turn  out  that  there  is  suf¬ 
ficient  stereoacuity  available  for  some  low-level  flight  tasks.  The 
evidence  presented  by  Regan  in  appendix  C  of  stereomotion  processing 
suggests  chc  need  for  binocular  displays  to  fully  portray  all  of  the 


information  produced  by  observer  motion;  but  for  this,  too,  it  needs  to 
be  demonstrated  that  stereomotion  acuity  is  present  even  at  the  dis¬ 
tance  typically  employed  in  low-level  flight. 

While  a  stereoscopic  display  in  a  helicopter  simulator  may  be 
useful  or  even  important,  accurate  display  of  surface  features  of 
helicopter  flying  is  critical.  When  so  much  of  the  flying  is  at  very 
low  speeds,  and  often  at  no  speed  at  all,  movement  produced  changes 
are  unavailable  or  uninformative.  In  the  absence  of  a  stereo  display, 
the  only  information  will  come  from  the  simulation  of  perspective 
transformations  of  light  reflected  surface  features;  and  if  the  fidel¬ 
ity  of  those  surface  features  is  poor,  if  only  a  few  features  are 
present,  or  if  their  resolution  is  inadequate,  simulator  performance 
will  necessarily  be  unrepresentative  of  actual  flying  performance. 

This  analysis  suggests  quite  different  visual  specifications  for 
simulation  at  low-level  flight.  The  most  useful  source  of  visual  in¬ 
formation  arises  from  observer-produced  motion.  Surface  details  are 
still  important  in  the  simulation,  but  since  the  pilot  is  capable  of 
moving  and  maneuvering  so  rapidly,  it  is  the  motion  information  that 
is  so  predominant.  Thus,  improving  fidelity  of  surface  texture,  es¬ 
pecially  of  fine-grain  texture,  may  not  improve  low-level  flight  simu¬ 
lation  at  all,  whereas  providing  visual  information  that  is  easily 
processed  for  relative  displacements  over  time  across  the  retina  would 
seem  to  be  essential.  Westra,  Simon,  Collyer,  and  Chambers  (1981) 
reached  the  same  conclusion  based  upon  their  experimental  manipulation 
of  a  number  of  surface  and  motion  variables  for  carrier  landing  flight 
performance,  also  a  low-level  flight  task. 

Given  the  focus  of  this  report  on  low-level  flight ,  the  purpose 
of  this  brief  paper  is  to  examine  some  of  the  information  sources  that 
are  produced  by  the  pilot's  movement  through  space.  I  especially  want 
to  single  out  one  subclass  of  these  sources,  that  based  on  optic  flow 
patterns.  This  focus  has  potential  interest  because  attending  to  such 
information  sources  may  actually  reduce  rather  than  increase  the  de¬ 
mands  on  computing  capacity,  and  because  they  are  likely  to  be  especi¬ 
ally  informative  of  the  layout  of  visual  space.  Some  suggestive  ex¬ 
periments  to  be  done  on  a  wide-angle  flight  simulator  are  sketched  at 
the  end. 


A  Division  of  Labor — Central  and  Peripheral  Vision 

We  tend  to  look  directly  at  the  important  sources  of  information 
in  visual  scenes  because  doing  so  images  that  information  on  the  cen¬ 
tral  high-resolving  part  of  the  retina.  But  the  fo^ea  can  cover  only 
about  one-ten-thousandth  of  the  area  of  a  visual  scene  at  any  one 
moment,  the  rest  falling  on  the  peripheral  retina  of  lower  resolving 
power.  While  we  have  learned  a  great  deal  about  information  process¬ 
ing  in  central  vision,  much  less  is  known  about  how  peripheral  stimu¬ 
lation  is  impaired  when  only  central  vision  is  permitted.  Particu¬ 
larly,  it  seems  likely  that  tasks  involving  orientation  in  visual 
space  and  especially  locomotion  through  visual  space  depend  upon 
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information  from  the  full  scene  information  available  only  to  the 
peripheral  retina. 1 

While  the  above  comments  are  couched  in  a  somewhat  intuitive  form, 
several  recent  theoretical  developments  have  helped  formali2e  the  in¬ 
tuitions.  The  most  important  of  these  is  the  notion  that  visual  func¬ 
tioning  may  be  served  by  two  somewhat  independent  visual  processing 
systems,  one  concerned  with  object  recognition  and  identification — 
the  "what"  of  perceptual  experience;  and  the  other  concerned  with 
spatial  localization  and  orientation — the  "where"  question  (e.g., 
Schneider,  1969;  Held,  1970).  Such  a  division  of  functioning  coin¬ 
cides  closely  wi  hi  the  differential  sensitivities  of  the  central  versus 
■'eripheral  retina,  and  with  the  differential  degree  of  optical  resolu¬ 
tion  over  these  areas.  It  also  relates  to  differential  loss  of  func¬ 
tion  that  occurs  with  disturbance  to  different  brain  structures  (e.g., 
Poppel  et  al . ,  1973),  suggesting  some  independence  to  processing  of 
what  and  where  information.  Finally,  it  has  been  shown  (e.g.,  Leibo- 
witz  et  al.,  1979)  that  while  questions  of  identification  are  criti¬ 
cally  dependent  upon  both  stimulus  energy  and  the  optical  quality  of 
the  retinal  image,  spatial  orientation  processing  is  relatively  inde¬ 
pendent  of  energy  and  image  of  quality  as  long  as  the  scene  is  visible 
at  all. 

A  substantial  part  of  the  task  facing  a  pilot  during  low-level 
flight  concerns  having  precise  visual  information  about  the  location 
of  the  ground  in  relation  to  the  plane  as  it  changes  from  moment  to 
moment.  From  the  distinction  presented  above,  it  would  seem  likely 
that  such  information  is  imaged  over  the  entire  retinal  surface  and 
not  concentrated  only  over  the  fovea,  and  is  not  dependent  upon  either 
high  optical  resolution  or  intense  energy.  Both  of  these  conclusions, 
if  true,  have  obvious  implications  for  the  design  of  low-level  flight 
simulation  displays.  * 


Spatial  Orientation  Information  Available 
to  a  Moving  Perceiver 

Perspective  geometry  provides  a  description  of  changes  in  the  pro¬ 
jection  of  light  to  an  observer  reflected  from  objects  and  surfaces 
as  the  observer  moves  (see  Haber,  1979,  for  a  brief  review).  The  sim¬ 
plest  circumstance,  called  motion  parallax,  is  when  there  are  only  two 
points  in  the  scene  reflecting  iight:  in  such  cases,  the  light  re¬ 
flected  from  the  two  points  shifts  position  on  the  retina  in  propor¬ 
tion  to  their  relative  distance  from  the  observer.  Thus,  the  amount 
of  relative  change  over  time  on  the  retina  can  provide  information 
about  relative  distance  from  the  observer — information  that  is  not 
available  to  a  stationary  observer.  While  motion  parallax  is  a  simple 


Empirical  investigations  have  not  shown  much  effect  of  display  field 
of  view  on  tasks  such  as  landing.  In  contrast,  with  some  tasks,  such 
as  aerobatic  maneuvers,  field  of  view  has  been  found  to  affect  per¬ 
formance  (see  Hennessy  et  al.,  1980). 
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circumstance,  and  is  considered  one  of  the  basic  monocular  cues  to 
depth,  much  more  interest  has  been  focused  on  its  generalized  exten¬ 
sion,  in  which  all  of  the  projected  points  of  light  move  relative  to 
one  another  as  an  observer  moves  through  a  visual  scene.  Gibson 
(1950)  referred  to  this  as  motion  perspective  and  emphasized  that  the 
information  covers  the  entire  retinal  projection  surface  and  is  not 
restricted  to  merely  a  few  points  of  light  in  relative  motion.  Thus, 
when  an  observer  moves,  a  continuous  gradient  of  perspective  is  pro¬ 
jected  onto  the  retina,  with  the  rate  of  change  at  each  point  provid¬ 
ing  a  mapping  of  the  contours  of  the  terrain  being  traversed.  Gordon 
(1965)  and  more  recently  Koenderink  and  van  Doom  (1975,  1976)  have 
provided  a  detailed  breakdown  of  the  geometrical  information  available 
at  a  projection  surface  to  a  moving  observer.  Nakayama  and  Loomis 
(1974)  have  carried  this  work  further  by  suggesting  the  kinds  of 
n-ural  mechanisms  that  could  encode  this  gradient — neural  mechanisms 
of  the  same  order  as  those  that  account  for  binocular  disparity  en¬ 
codings.  Ullman  (1979)  has  incorporated  these  features  with  a  more 
general  model  of  motion  perception,  and  Lee  and  Lishman  (1977)  have 
reported  some  psvchophvsical  experiments  on  the  visual  control  of 
locomotion  using  notion  perspective  information. 

One  aspect  of  motion  perspective  discussed  in  detail  by  Gibson 
and  others  since  refers  to  optic  flow  information.  When  looking 
ahead,  light  reflected  from  the  point  in  the  scene  toward  which  the 
observer  is  moving  remains  centered  on  the  retina,  only  expanding  in 
size  as  it  gets  closer.  All  other  reflected  points  flow  across  the 
retina  toward  the  periphery,  with  a  relative  rate  of  flow  proportional 
to  the  distance  of  the  reflecting  surface  to  the  observer.  in  this 
way,  the  flow  pattern  represents  the  layout  of  the  scene,  with  the 
aiming  point  of  the  moving  obsert’er  being  indicated  by  a  zero  flow 
vector.  Since  observers  usually  look  at  that  toward  which  they  move, 
this  means  that  information  about  the  direction  of  observer  motion  is 
available  to  central  vision.  But  the  flowing  part  of  the  flow  pattern 
covers  the  entire  retina,  and  not  only  can  specify  direction  and  ve¬ 
locity  of  the  observer,  but  more  important,  the  location  and  irregu¬ 
larities  of  the  terrain.  None  of  this  specification  need  be  projected 
to  central  vision.  It  is  likely  that  such  optic  flow  patterns  are 
the  most  important  source  of  information  about  general  terrain  char¬ 
acteristics  and  the  observer's  orientation  and  location  over  that 
terrain.  Because  optic  flow  does  not  depend  upon  central  vision,  it 
may  be  handled  entirely  by  the  "where"  system,  and  if  so,  flow  can  be 
represented  by  a  simulato*-  display  with  low-resolution,  low-intensity 
elements . 

I  have  three  further  notes  before  suggesting  some  experiments. 
When  an  observer  moves  relative  to  the  terrain,  a  system  with  high 
temporal  resolution  is  capable  of  representing  each  projected  poj.nt 
of  light  as  a  unique  point.  The  human  visual  system,  however,  has  a 
relatively  low  temporal  sensitivity,  so  that  a  moving  point  of  light 
is  perceived  as  a  streak  or  blur  line  (Smith,  1969).  Much  of  the  op¬ 
tic  flow  has  to  be  streaks,  especially  in  the  periphery  of  the  retina. 
Harrington  et  al.  (1980)  have  shown  that  perceivers  are  able  to  judge 


orientation  quite  accurately  from  optic  flow  that  was  entirely  blurred, 
even  out  of  75  degrees  from  the  fovea. 

My  second  note  concerns  the  effects  of  psychological  or  physio¬ 
logical  stress  on  the  processing  of  "what"  and  "where"  information, 
and  especially  on  the  relative  weights  placed  on  central  and  periphery 
information.  Mackworth  (1965,  1976)  reported  studies  that  suggested 
that  under  stress  or  high  information  loads,  identification  process¬ 
ing  becomes  tunneled,  so  that  information  picked  up  peripherally  is 
less  adequately  processed.  His  studies  did  not  investigate  location 
or  orientation  information  processing,  so  it  is  not  clear  whether 
stress  produces  tunnel  vision  for  "where"  as  well  as  "what"  informa¬ 
tion;  though  it  is  possible  that  the  periphery  does  not  suffer  any 
functional  loss  for  processing  of  optic  flow  information.  It  is  im¬ 
portant  to  know  how  the  details  contained  in  the  flow  are  processed: 

(a)  whic  i  provide  information  about  the  objects  on  the  terrain,  as  com¬ 
pared  to  how  the  flow  itself  is  processed,  and  (b)  which  specify  the 
orientation  of  the  terrain. 

My  third  point  is  a  more  general  one.  We  as  yet  do  not  know  all 
of  the  sources  of  "isual  information  that  are  available,  and  even  of 
those,  we  do  not  know  which  ones  are  used  in  what  contexts.  Thus, 
one  of  the  goals  of  research  programs  on  space  perception,  even  one 
focused  on  simulation  training,  has  to  be  to  examine  the  psychophysi¬ 
cal  relationships  between  sources  of  information  and  subsequent  per¬ 
ceptual  performance.  This  need  is  powerfully  illustrated  by  the  recent 
work  of  Owen  (e.g.,  Owen  et  al . ,  1981a, b) ,  who  has  shown  that  it  mat¬ 
ters  greatly  whether  the  visual  variables  are  defined  in  terms  of 
visual  information  specifying  environmental  surfaces  or  in  terms  of 
specifying  the  relations  of  the  perceiver  to  those  surfaces.  In 
Owen's  experiments,  it  turns  out  that  the  latter  specifications  are 
the  ones  which  predict  performance,  suggesting  that  the  psychophysical 
relationships  include  the  perceiver  in  the  equation.  The  important 
point  is  that  there  are  many  ways  to  define  and  specify  each  of  many 
different  optical  variables,  not  all  of  which  are  equivalent.  Only 
by  psychophysical  research  can  we  determine  which  variables  have  what 
properties.  It  cannot  be  done  simply  by  a  geometric  analysis  in  an 
arm  chair.  " 


Some  Suggested  Experiments 

With  these  questions  and  assumptions  in  mind,  I  have  sketched 
out  some  research  problems  that  should  be  addressed  and,  in  some 
cases,  have  provided  more  details  as  to  procedure.  While  my  focus 
is  primarily  on  the  use  of  visual  information  that  falls  on  the 
peripheral  retina  during  low-level  flight  and  therefore  in  how  to 
present  such  information  in  simulation  of  low-level  flight,  expanding 
our  knowledge  in  this  area  will  correct  deficits  iri  current  theories 
of  visual  perception  and  could  be  of  great  practical  value  in  design¬ 
ing  the  arrangements  of  visual  displays  such  as  instrument  panels, 
monitors,  and  warning  devices;  and  in  designing  general  visual  simu¬ 
lation  equipment  and  the  programs  for  displaying  simulated  scenes, 
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not  only  for  low-level  flight  but  for  tasks  like  driver  training  and 
for  improved  methods  of  training  target  detection. 

What  is  needed  are  studies  directed  at  demonstrating  the  aspects 
of  the  visual  display  actually  used  in  various  locomotion  tasks. 

Since  we  need  to  know  this  within  the  context  of  low-level  flight 
simulation,  such  studies  require  the  use  of  a  wide  f ield-of-view 
simulator  programmed  for  low-level  flight.  For  some  of  the  experi¬ 
ments  sketched  below,  an  additional  closed  loop  control  is  needed, 
so  that  the  monitored  direction  of  gaze  of  the  subject's  eyes  is  used 
to  vary  the  content  of  the  display.  Hence,  when  looking  straight 
ahead,  the  details  of  the  optic  flow  pattern  seen  through  the  side 
windows  would  be  of  low  resolution,  but  if  the  pilot  turned  his  head 
to  look  to  the  side,  the  resolution  would  sharpen  to  correspond  to 
the  improved  acuity  of  central  vision.  Such  on-line  monitoring  of 
gaze  direction  and  closed  loop  control  of  display  resolution  is  not 
necessary  for  all  of  the  experiments,  but  it  is  critical  for  some 
that  should  be  done . 

The  general  procedure  involves  pilot-subjects  (perhaps  including 
subjects  with  little  flight  experience)  flying  in  a  wide-view  simu¬ 
lator  on  various  low-level  flight  tasks,  such  as  terrain  avoidance 
while  maintaining  a  limited  altitude,  reconnaisance  and  target  acqui¬ 
sition,  or  more  specific  tasks  such  as  flying  a  prescribed  course  or 
maintaining  a  fixed  altitude  above  the  ground.  Given  these  types  of 
tasks,  various  stimulus  changes  could  be  evaluated  as  to  their  effect 
on  flight  performance. 

One  set  of  studies  involving  stimulus  manipulation  might  provide 
a  mislocation  of  the  zero  point  of  the  optic  flow  vector,  so  that  the 
plane's  flight  path  was  not  always  aimed  at  the  zero  point,  even  though 
the  rest  of  the  flow  pattern  was  correct.  If  pilots  use  this  informa¬ 
tion,  substantial  disruption  in  flight  performance  (and  perhaps  dis¬ 
orientation  and  discomfort)  should  occur.  A  related  manipulation 
might  occlude  the  zero  point  in  the  flow  pattern  so  the  pilot  could 
see  it,  though  he  could  see  all  else  in  the  scene.  Further,  along 
the  same  lines,  we  could  manipulate  how  far  ahead  of  the  plane  optic 
flow  information  is  useful  in  relation  to  velocity.  When  flying  at 
high  speed,  pilots  gain  little  useful  information  directly  in  front 
of  the  plane.  In  effect,  the  pilot  has  to  focus  well  ahead.  This 
tunneling  of  vision  produces  more  reliance  on  optic  flow  because  less 
detailed  pattern  information  can  be  picked  up. 

A  different  set  of  studies  should  look  at  the  content  of  the 
flow  pattern  itself. 

Conceivably  the  flow  is  processed  entirely  in  the  peripheral 
retina,  and  conceivably  it  is  handled  by  neural  circuits  concerned 
only  with  motion  and  not  with  pattern  or  edges.  In  that  case,  it 
would  be  less  important  how  flow  is  represented  in  the  display,  and 
:onsiderably  lower  resolution  could  be  used  without  loss  of  perfor¬ 
mance.  Again,  the  same  sorts  of  tasks  can  be  used  to  evaluate  per¬ 
formance  as  a  function  of  the  content  and  the  resolution  of  the  flow. 

A  third  set  of  studies  could  look  at  an  interaction  of  flow  pat¬ 
terning  with  detection  of  specific  targets  as  the  flow  passes.  If 
flow  is  merely  relative  movement,  how  do  we  also  detect  the  presence 
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of  a  sought-after  target  object?  When  looking  straight  ahead  at  the 
zero  vector  point,  what  are  the  minimal  size,  contrast,  color,  and 
context  characteristics  of  objects  displayed  in  the  optic  flow  neces¬ 
sary  for  detection  as  they  stream  past  the  observer?  Presumably,  a 
sought-after  or  surprising  object,  once  detected,  causes  an  eye  move¬ 
ment  toward  it  away  from  the  zero  point,  but  what  does  it  take  to 
produce  detection  and  reorientation? 

These  studies  constitute  only  a  small  subset  of  possible  ones 
that  could  be  done  on  flow  and  peripheral  information  pickup.  But 
they  are  a  start,  and  they  can  be  easily  expanded  once  the  programs 
and  procedures  are  set  up.  None  of  these  studies  are  ground-breaking 
in  a  theoretical  sense,  though  they  will  allow  us  to  test  some  of  our 
hallowed  theories.  But  they  provide  needed  data  for  design  of  flight 
simulation  displays,  as  well  as  general  visual  training  procedures. 
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APPENDIX  E 


DISPLAY  DESIGN  VARIABLES  PERTINENT  TO  LOW-LEVEL 
FLIGHT  SIMULATION 

Harry  L.  Snyder 


Introduction 

The  main  body  and  other  appendixes  of  this  report  contain  analy¬ 
ses  of  known  difficulties  and  problems  in  the  visual  simulation  image 
content  of  low-level  flight.  Issues  of  pertinent  visual  scene  param¬ 
eters  are  addressed,  the  importance  of  many  visual  parameters  to  pilot 
training  and  performance  is  noted,  and  suggested  experimental  activi¬ 
ties  to  define  better  the  information  content  of  the  visual  simulation 
are  described.  This  paper,  on  the  other  hand,  takes  a  different  ap¬ 
proach,  one  complementary  to  the  others.  It  does  not  address  the  re¬ 
quirements  for  information  content  of  the  visual  scene,  but  rather 
addresses  the  requirements  for  the  display  generation  equipment,  in 
visually  relevant  terms. 

This  appendix  therefore  serves  two  important  purposes.  First,  it 
notes  the  pertinent  hardware/software  design  requirements  for  visually 
compatible  information  presentation  if^  the  information  is  adequately 
defined  and  described  in  usable  parametric  terms.  Second,  it  addresses 
some  of  the  technologically  limiting  characteristics  of  visual  dis¬ 
plays,  thereby  noting  the  limits  of  the  visual  information  which  can 
be  presented. 

In  the  final  analysis,  if  visual  information  presentation  require¬ 
ments  are  found,  through  analysis  or  empirical  experimentation,  to  ex¬ 
ceed  current  display  generation  technology,  then  research  and  develop¬ 
ment  requirements  for  simulation  technology  can  be  defined  in  a  useful 
manner.  On  the  other  hand,  if  information  requirements  are  far  less 
than  those  of  which  the  simulation  equipment  is  capable,  then  potential 
cost  savings  and  complexity  reduction  can  be  achieved.  In  either  event, 
it  is  important  to  understand  the  relationships  between  visually  perti¬ 
nent  variables  and  measurable  characteristics  of  the  simulation 
equipment . 

Finally,  this  appendix  addresses  briefly  the  issue  of  measurement 
and  verification  of  the  visual  display  in  photometric  and  geometric 
ways  directly  pertinent  to  the  visual  characteristics  of  importance. 
Display  technology  has  advanced  rapidly,  at  a  more  rapid  rate  than  the 
metrology  needed  to  define  and  refine  its  image-producing  specifica¬ 
tions.  While  some  work  has  been  done  recently  to  recommend  those 
techniques  and  criteria  for  adequate  and  accurate  display  quantification. 
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universal  application  of  these  techniques  is  far  from  prevalent.  Thus, 
the  relationships  between  some  of  these  measurement  techniques  and 
vision-related  variables  are  summarized  briefly. 

Display  Variables  Critical  to  Visual  Simulation 

This  section  describes  those  display  variables  considered  to  be 
critical  to  the  presentation  of  visual  information  in  low-level  flight. 
The  format,  density,  and  content  of  the  information  to  be  displayed  is 
described  elsewhere  in  this  report.  This  section  assumes  the  need  for 
the  visually  compatible  presentation  of  various  types  of  information, 
and  suggests  minimal  relationships  which  must  hold  for  adequate  pre¬ 
sentation  of  that  information.  Wherever  possible,  known  data  and  ex¬ 
perimental  results  will  be  referenced. 


Display  Field  of  View 

The  field  of  view  (FOV)  requirements  for  visual  simulation  have 
been  amply  shown  to  be  task  dependent.  Take-off  and  landing  tasks  ap¬ 
pear  to  require  only  a  limited  FOV,  on  the  order  of  50  to  60  degrees 
or  less.  On  the  other  hand,  air-to-air  combat,  and  by  analogy  low- 
level  flight,  requires  visual  scanning  by  the  pilot  and  foveal  acqui¬ 
sition  of  high  spatial  frequency  information  from  various  parts  of 
the  visual  field  typically  available  through  the  cockpit  canopy. 

Studies  have  included  parametric  investigation  of  FOVs  as  large  as 
180  degrees.  Although  statistical  results  have  been  limited,  it  ap¬ 
pears  that  the  larger  (e.g.,  greater  than  150  degrees)  FOV  is  needed 
for  reconnaissance,  low-level  navigation,  and  target  acquisition  ac¬ 
tivities,  and  perhaps  for  terrain  avoidance. 

Translation  of  this  requirement  into  realistic  hardware  presents 
several  problems.  First,  a  single  image,  from  a  single  projection  de¬ 
vice,  to  cover  at  least  150  degrees  is  limited  to  either  low-resolution 
electronic  imaging  or  higher-resolution  film-projection  devices.  For 
example,  the  JTF-2  visual  simulator  had  a  field  of  view  of  160  by  60 
degrees,  used  a  70  mm  film  format,  and  had  a  limiting  (high-contrast) 
resolution  on  the  order  of  3  arc  minutes  in  the  center  of  the  FOV. 
Single  CRT  projection  devices  will  have  a  maximum  resolution  on  the 
order  of  1,200  picture  elements  (pixels)  per  display  width,  and  there¬ 
fore  no  better  than  8  arc  minutes  limiting  resolution.  For  this 
reason,  the  typical  approach  has  been  toward  butted  images  from  sev¬ 
eral  CRTs,  with  the  attendant  problem  of  edge  discontinuities  or  image 
gaps  due  to  registration  errors  between  adjacent  images.  (The  ad¬ 
jacency  or  segmentation  problem  is  addressed  below.)  High-resolution 
elements  (e.g.,  targets)  can  also  be  inserted  from  a  second  projector, 
and  high-resolution  "windows"  in  the  total  field  can  be  slaved  to  the 
pilot's  head  position  to  improve  resolution  in  a  part  of  the  display. 

At  the  present  time,  the  designer  has  a  complex  trade-off  to  make 
between  FOV  and  resolution,  increases  in  one  generally  causing  de¬ 
creases  in  the  other.  Since  no  quantitative  data  appear  to  exist  on 
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the  effects  of  FOV  on  low-level  flight  control  or  navigation,  it  would 
appear  that  parametric  research  dedicated  toward  this  determination  is 
extremely  important.  Preceding  such  experimentation,  one  should  con¬ 
duct  thorough  analyses  of  the  task  and  information  requirements  of 
the  pilot  in  the  mission  phases  of  interest.  As  Semple  et  al .  (1980: 
79-80)  pointed  out, 

the  value  of  a  wide  FOV  may  not  have  been  apparent  in  the 
studies  reviewed  because  the  tasks  did  not  specifically 
demand  use  of  more  than  a  central  FOV.  For  example,  air 
to  air  combat  obviously  requires  a  wide  FOV.  Therefore 
the  decision  on  whether  a  wide  or  narrow  FOV  is  necessary 
for  training  purposes  must  take  into  account  specific  task 
requirements  and  not  be  based  solely  on  possible  unwar¬ 
ranted  generalizations  from  research  results  .  .  .  from 
a  limited  number  of  flying  tasks. 

The  low-level  navigation  and  target  acquisition  task  is  not  among  those 
which  has  been  parametrically  investigated,  and  therefore  requires 
careful  research  on  FOV  trade-offs. 


Dynamic  (Luminance)  Range 

Projection  systems  have  a  finite  amount  of  luminous  energy  that 
can  be  distributed  across  the  FOV.  Increases  in  the  FOV,  obtained 
optically,  will  result  in  proportionately  lower  luminance  levels,  as¬ 
suming  screen  gains  and  optical  efficiency  to  be  approximately  con¬ 
stant.  The  only  exception  to  this  generalization  is  in  the  use  of 
multiple  CRTs  or  other  segmented  means  of  controlling  the  image  lumi¬ 
nance  per  screen  segment  by  increasing  the  number  of  sources  rather 
than  spreading  the  energy  from  a  single  source  over  a  wider  FOV. 

While  the  observer  will  visually  adapt  to  the  mean  luminance 
level  of  the  entire  display,  assuming  scanning  behavior  such  as  to 
fixate  foveally  the  overall  field,  reductions  in  luminance  range  will 
necessarily  reduce  the  number  of  discriminable  levels  of  luminance. 
More  importantly,  the  contrast  sensitivity  of  the  observer  will  de¬ 
crease  with  decreases  in  luminance,  thereby  further  reducing  the  ob¬ 
server's  discrimination  of  luminance  differences  and  decreasing  his 
cutoff  spatial  frequency,  resulting  in  a  loss  of  effective  acuity. 

It  would  appear  that  the  attenuation  of  luminance  range  is  im¬ 
portant  in  that  the  contrast  sensitivity  function  (CSF)  shift  is 
critical  to  the  perception  of  small  details,  and  perhaps  to  the  per¬ 
ception  of  depth  relationships.  Thus,  luminance  requirements  can  be 
stated  only  once  we  know  the  requirements  for  spatial  information 
presentation  (resolution)  of  the  image.  Experiments  to  determine 
the  resolution  requirements  are  considered  important  for  this  as 
well  as  other  reasons. 

Existing  research  results  permit  us  to  recommend  a  limiting 
resolution  of  no  more  than  3  arc  minutes  for  such  activities  as 
target  acquisition  in  an  air-to-ground  search  (Humes  and 
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Bauerschmidt ,  1968).  The  author  knows  of  no  comparable  data  for  per¬ 
formance  effects  attributable  to  resolution  in  tasks  such  as  low-level 
navigation  and  terrain  following.  Again,  a  task  analysis  leading  to 
information  requirements  might  help.  Other  experimental  techniques 
based  upon  scene  spatial  filtering  and  observer  static  target  and  ob¬ 
ject  recognition  might  also  be  helpful.  Simplified  closed-loop  flight 
control  simulations  with  varying  image  resolution  may  also  serve  to 
provide  useful  information  on  the  requirements  for  display  resolution 
and  therefore  dynamic  range. 


MTF 

The  preceding  discussion  related  resolution  requirements  to  dy¬ 
namic  range.  A  more  useful  measure,  which  combines  both  luminance 
range  and  resolution  is  the  modulation  transfer  function,  or  MTF. 

In  this  approach,  the  output  modulation  (or  contrast)  of  the  display 
is  determined  for  all  spatial  frequencies,  assuming  a  constant  input 
modulation  (M) .  Thus,  the  MTF  specifies  the  modulation  the  display 
system  is  capable  of  delivering  at  any  spatial  frequency,  assuming  a 
unity  modulation  input.  The  engineering  convention  of  describing 
"shades  of  gray"  as  20.5  increments  in  luminance  can  be  converted  to 
equivalent  modulation  by  the  following  formula: 

N  (shades  of  gray)  =  1  +  loS  ( (1 

log  2U-5 

In  recent  years  it  has  become  recognized  that  the  specification 
of  an  MTF  is  more  meaningful  and  pertinent  than  the  specification  of 
merely  a  limiting  resolution.  The  "limiting  resolution"  of  an  imag¬ 
ing  system  is,  in  actuality,  the  spatial  frequency  at  which  the  MTF 
of  the  display  system  crosses  the  CSF  of  the  observer  (Snyder,  1973). 
More  importantly,  the  MTF  also  describes  the  ability  of  the  display 
system  to  render  high  contrast  inputs  faithfully  at  spatial  frequen¬ 
cies  below  the  limiting  resolution.  As  such,  the  MTF  describes  the 
ability  of  the  imaging  system  to  faithfully  reproduce  input  contrasts 
at  all  usable  spatial  frequencies  of  the  system,  whereas  the  limiting 
resolution  specifies  only  the  spatial  frequency  at  which  the  system 
ceases  to  provide  information  to  the  observer,  a  measure  of  little 
utility. 

Little  research  has  been  conducted  to  relate  observer  performance 
to  MTF  variations  for  many  tasks,  although  data  do  exist  for  tasks 
such  as  face  recognition  and  air-to-ground  target  acquisition  (Gutmann 
et  al.,  1979;  Snyder,  1974;  Snyder  et  al.,  1974).  Assuming  the  low- 
level  navigation  and  flight  control  tasks  to  be  reasonably  related  to 
air-to-ground  target  acquisition,  then  it  would  appear  that  a  display 
system  with  an  MTF  at  least  equal  to  that  shown  in  figure  E-l  would 
be  desirable.  Unfortunately,  as  figure  E-l  also  illustrates,  current 
technology  is  far  from  this  capability.  Research  of  a  more  directly 
applicable  nature,  using  flight  control  rather  than  target  acquisi¬ 
tion  as  the  dependent  measure,  is  clearly  indicated. 
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FIGURE  E-l  Recommended  minimal  display  system  MTF  for  low-altitude  simulation 
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Refresh  Rate 

Computer-generated  displays  are  of  finite  bandwidth,  and  this 
bandwidth  must  be  allocated  between  the  number  of  displayed  picture 
elements  (pixels)  and  the  refresh  rate.  It  is  often  feasible  to 
reduce  the  refresh  rate  by  using  longer  persistence  display  surfaces, 
thereby  achieving  greater  spatial  resolution.  However,  with  dynamic 
motion  typical  of  low-altitude  flight,  longer  persistence  displays 
would  be  subject  to  blur  or  streaming.  Thus,  a  careful  trade-off  is 
needed  between  spatial  resolution  and  temporal  resolution  (refresh 
rate).  This  trade-off  also  exists  in  film  projection  systems,  where 
the  shutter  angle  of  the  cine  projector,  in  conjunction  with  its 
frame  rate,  must  be  selected  to  avoid  image  jump  or  strobing. 

The  refresh  rate  is  set  by  both  flicker  fusion  thresholds  and 
the  image  motion  rate.  As  image  motion  increases,  typically  at  the 
bottom  center  portion  of  the  display  as  a  limiting  condition,  inade¬ 
quate  refresh  rates  yield  image  breakup  or  a  perception  of  image 
strobing  (Snyder  et  al . ,  1966).  Solutions  to  the  strobing  problem 
are  to  increase  the  refresh  rate,  permit  blurring,  or  decrease  the 
angular  rate  of  the  image  by  decreasing  the  look-down  angle.  These 
trade-offs  are  quantitative  and  established  in  terms  of  the  strobing 
phenomenon  (Snyder  et  al . ,  1966);  further,  the  minimum  refresh  rate 
to  avoid  flicker  can  be  specified  by  a  Fourier  analysis  of  the  tem¬ 
poral  luminance  distribution,  which  includes  the  refresh  rate  con¬ 
volved  with  the  persistence  of  the  display  system  (Snyder,  1980). 
Reductions  in  refresh  rate  are  helpful  in  bandwidth  allocation.  A 
60-Hz  rate,  while  standard,  may  not  be  needed. 

In  general,  then,  the  needed  specification  for  refresh  rate  de¬ 
pends  largely  upon  knowledge  of  the  field  of  view,  the  maximum  simu¬ 
lated  aircraft  velocity  and  altitude  (to  obtain  V/H) ,  and  the  number 
of  pixels  in  the  display  requiring  refresh  every  frame  rate.  No  re¬ 
search  seems  warranted  on  this  parameter  except  for  verification 
studies  once  the  FOV  and  resolution/MTF  requirements  are  set. 


Information  Update  Rate 

The  display  refresh  r  te  is  established  to  avoid  flicker,  and  must 
also  be  compatible  with  the  need  to  eliminate  strobing,  as  described 
above.  At  the  same  time,  there  is  often  a  need  to  refresh  the  display 
to  change  the  state  of  information  depicted  on  the  display.  Some  in¬ 
formation  rate  updates  are  required  by  the  strobing  problem,  but 
others  are  needed  to  achieve  sufficiently  accurate  information  place¬ 
ment  in  the  field,  even  though  strobing  would  not  occur  with  a  lower 
update  rate.  Careful  analysis  of  the  rate  of  movement  of  displayed 
elements,  along  with  the  placement  accuracy  required,  both  spatially 
and  temporally,  is  needed  for  the  low-altitude  flight  regime.  This 
analysis,  in  conjunction  with  the  information  requirements  work  de¬ 
scribed  elsewhere,  should  produce  information  update  rates  compatible 
with  the  pilot's  visual  needs. 


Virtual  Versus  Real  Image 

Two  approaches  are  popular  in  visual  flight  simulation.  In  one, 
the  image  is  created  on  a  display  surface  such  as  a  CRT  and  viewed 
through  a  collimating  lens  to  produce  an  image  at  optical  infinity. 
With  this  approach,  the  pilot  can  move  his  head  in  translation  with¬ 
out  relative  image  motion;  in  addition,  the  accommodation  of  the  eye 
is  comparable  to  that  required  in  actual  flight  to  view  the  real 
world.  In  this  mode,  the  pilot  must  shift  accommodation  realistic¬ 
ally  between  the  visual  scene  of  the  projected  display  and  the  in¬ 
cockpit  information,  creating  thereby  appropriate  visual  loading  and 
accommodation  time  lags. 

The  second  approach  largely  ignores  this  problem  of  accommoda¬ 
tion  shift  between  in-cockpit  tasks  and  optical  infinity,  and  pro¬ 
jects  a  real  image  on  a  (typically)  spherical  screen  in  front  of  the 
cockpit.  The  radius  of  curvature  of  such  screens  is  usually  on  the 
order  of  6  to  15  feet,  thus  requiring  some  accommodation  shift  between 
the  cockpit  displays  and  the  projected  real  world;  however,  the  ad¬ 
vantages  of  this  type  of  simulation  are  freedom  of  distortion  and 
separation  introduced  by  the  segmented  collimating  optics  or  separate 
display  surface. 

It  must  be  realized  that  noncollimated  display  of  scene  informa¬ 
tion  that  would  normally  exist  at  optical  infinity  may  lead  to  mis¬ 
perception  of  the  size  of  objects  (Leibowitz  et  al.,  1972;  Roscoe, 
1951).  In  addition,  movement  of  the  pilot's  head  with  respect  to 
noncollimated  images  may  result  in  misjudged  distance  perception 
(Gogel  and  Teitz ,  1973),  although  it  is  likely  that  an  image  distance 
in  excess  of  10  feet  will  minimize  this  effect. 

Whether  the  collimated  display  has  any  real  advantage,  particu¬ 
larly  for  training,  over  the  noncollimated,  real-image  display  is  an 
empirical  question.  Certainly,  if  the  noncollimated  display  can  avoid 
the  image  segmentation  typical  of  many  multiple-image,  pancake-window 
optics  used  in  collimated,  wide-FOV  displays,  then  the  advantage  may 
well  rest  with  the  noncoil >*  ed  display.  It  would  appear,  as  sug¬ 
gested  by  Semple  et  al.  (  ,  that  the  decision  may  well  be  a  mat¬ 

ter  of  economics  rather  than  performance,  although  this  writer  recom¬ 
mends  a  direct  experimental  comparison  between  the  two,  using  the  same 
tasks  and  imagery,  to  determine  if  any  significant  difference  exists. 


Image  Geometric  Linearity 

Perfectly  collimated  images  will  produce  geometrically  linear 
displays  if  the  display  generator  is  geometrically  perfect.  All  too 
typically,  however,  the  edges  of  the  display  suffer  from  slight  non- 
collimation,  exhibiting  a  small  amount  of  "swim"  in  the  corners. 

This  can  be  distracting  to  the  observer,  especially  if  the  image 
corners  are  butted  with  other  images.  The  same  problem  can  exist  in 
a  noncollimated  image  if  sufficient  care  is  not  taken  with  the  ana- 
morphic  optics  (among  others)  to  correct  for  screen  curvature,  center 
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of  curvature,  etc.  (The  subject's  head  and  the  projector  cannot  both 
occupy  the  center  of  curvature  position!) 

A  critical  question  is  the  extent  to  which  noncollimat ion  or 
geometric  nonlinearity  may  cause  performance  or  training  differences 
by  pilots.  There  are  no  data  on  the  effects  of  display  nonlinearity 
on  even  simple  tasks  such  as  object  recognition  or  text  reading,  let 
alone  on  pilot  performance  during  low  altitude  flight  (Snyder,  1980). 
Such  data  are  needed,  but  are  probably  not  as  critically  related  to 
pilot  performance  or  training  efficiency  as  are  other  variables  dis¬ 
cussed  in  this  appendix. 


Image  Segmentation 

The  problem  of  alignment  and  separation  of  multiple-image  dis¬ 
plays  was  discussed  in  a  literature  review  and  analysis  report  by 
Kraft  et  al.  (1980),  who  recommended  experimental  investigation  of 
the  segmentation  and  alignment  problem.  In  a  subsequent  investiga¬ 
tion,  Kraft  and  Anderson  (1980)  determined  the  ability  of  U.S.  Air 
Force  pilots  to  detect  joint-width  differences,  misalignments,  and 
rotations  among  image  portions.  This  study  was  conducted  with  static, 
daylight  scenes,  and  the  subjects  were  not  required  to  perform  any 
complex,  closed-loop  task  by  reference  to  the  scenes. 

Kraft  and  Anderson  (1980)  found  that  joint  widths  of  15  arc 
minutes  or  more  masked  vertical  displacement  errors  up  to  1.4  arc 
minutes.  They  also  found  that  joint  widths  of  less  than  15  arc  min¬ 
utes  led  to  more  critical  tolerances  of  rotational  errors. 

Whether  these  acute  judgments  by  their  subjects  would  be  upheld 
in  more  complex  tasks,  such  as  low-altitude  navigation  and  terrain 
following,  is  an  empirical  question.  It  would  seem  doubtful  that  the 
heavily  loaded  pilot  would  be  as  sensitive  to  such  small  errors  in 
the  visual  display  under  such  conditions.  Further,  it  may  well  be 
the  case  that  detection  of  small  image  segmentation  and  alignment 
errors  may  have  no  impact  cn  the  training  utility  of  the  simulator. 

At  the  present  time,  no  useful  data  exist  on  this  point. 


Jitter 

Many  CRT  displays  exhibit  substantial  vertical  jitter,  a  field- 
to-field  or  frame- to- frame  vertica.'  movement  in  the  image  caused  by 
instability  of  the  vertical  deflection  circuitry.  Unfortunately, 
such  jitter  is  often  greater  than  the  spot  size  of  the  display, 
thereby  causing  a  reduced  vertical  resolution  or  a  high-frequency- 
roll-off  of  the  MTF  in  excess  of  what  spot-size  limitations  would 
dictate . 

It  is  quite  feasible  that  this  jitter,  while  not  detectable  di¬ 
rectly,  causes  many  of  the  visual  fatigue  symptoms  experienced  by 
users  of  visual  display  units  (usually  CRTs)  in  office  environments. 
If  so,  it  is  also  reasonable  to  think  that  jitter  may  cause  visual 
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fatigue  (tiredness,  headaches,  etc,)  in  the  simulator  pilot  after  sub¬ 
stantial  time  in  the  simulator. 

While  jitter  is  measurable,  it  is  often  not  carefully  controlled 
in  CRT  deflection  design.  Thus,  many  solid  state,  flat  panel  dis¬ 
plays  are  judged  to  be  better  in  image  quality  by  some  observers  due 
to  their  lack  of  jitter. 

No  experimental  data  currently  exist  on  the  relationship  between 
jitter  amplitude  and  frequency  and  objective  performance  measures  of 
observers.  Further,  no  data  exist  on  the  effect  of  jitter  variables 
on  subjective  reports  of  visual  fatigue,  although  anecdotal  stories 
have  been  recorded  (Snyder,  1980).  A  parametric  study  of  the  effect 
of  jitter  on  visual  performance  is  clearly  warranted.  In  the  absence 
of  such  experimentation,  careful  measurement  and  control  of  jitter 
should  be  of  prime  concern  in  the  design  of  any  visual  simulation 
system. 


Color 


The  use  of  color  in  visual  simulation  for  training  has  been  in¬ 
vestigated  in  at  least  two  studies.  Chase  (1970)  investigated  the 
effect  of  color  on  training  for  approaches  and  landings,  and  obtained 
small  but  positive  differences  in  favor  of  color.  On  the  other  hand. 
Woodruff  (1979)  found  no  differences  in  either  learning  rates  or 
final  performance  using  color  versus  black-and-white  TV  displays. 
Similar  results  have  been  found  in  air-to-ground  target  acquisition 
tasks.  Snyder,  Greening,  and  Calhoun  (1964)  found  no  differences  in 
target  acquisition  for  color  versus  black-and-white  in  a  film  projec¬ 
tion  simulation,  while  Fowler  and  Jones  (1972)  found  no  differences 
in  an  air-to-ground  TV  simulation. 

It  should  be  noted  that  the  atmospheric  attenuation  of  color  is 
quite  rapid,  with  very  little  color  remaining  beyond  a  few  miles  of 
clear  atmosphere  (Middleton,  1952).  With  the  exception  of  simulation 
of  cultural  objects,  such  as  landing  and  r unway /taxiway  lights  which 
are  intrinsically  color  coded,  there  appears  to  be  little  need  for 
color  simulation,  although  pilots  report  that  a  simulator  display 
with  color  is  very  pleasing  (Semple  et  al.,  1980). 

It  appears  that  parametric  experimentation  on  the  requirements 
for  color  in  low-altitude  flight  Is  urgently  needed,  not  because  of 
the  aesthetics  involved,  but  rather  because  of  the  economics.  Image 
generation  in  color  is  usually  more  complex  and  expensive,  as  is  the 
design  of  optics  that  avoid  chromatic  distortion.  Further,  the  pos¬ 
sibility  of  false  depth  perception,  due  to  chromostereopsis ,  must  be 
avoided.  If  color  does  not  aid  in  either  performance  or  transfer  of 
training,  then  there  appears  to  be  no  noncosmetic  requirement  for 
such.  On  the  other  hand,  color  enhancement  might  provide  improved 
training  efficiency,  but  this  determination  is  yet  to  be  made. 
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Stereo  Imaging 

The  need  for  stereoscopic  display  for  flight  simulation  has  been 
considered  by  many,  but  never  implemented  or  evaluated  experimen¬ 
tally.  On  the  negative  side,  one  can  argue  that  most  of  the  visual 
information  pertinent  to  the  flight  environment  occurs  at  distances 
on  the  order  of  several  hundred  feet  or  more,  and  that  no  binocular 
cues  (e.g.,  convergence,  retinal  disparity)  or  accommodation  cues 
exist  at  tha..  ntical  distance.  Conversely,  the  existence  of  stereo 
channels  (see  akt  ■  ndix  C)  creates  some  concern  for  potential  distor¬ 
tion  of  the  visual  .  cene  without  presentation  of  stereoscopic  infor¬ 
mation  which  might  be  above  threshold  for  these  channels. 

While  this  issue  cannot  be  adequately  resolved  at  this  time  due 
to  lack  of  a  complete  understanding  of  the  relationship  between  stereo 
channels  and  low-altitude  flight  performance,  the  issue  certainly 
warrants  further  analysis  and  perhaps  experimentation  along  the  lines 
presented  in  appendix  C. 


Measurement  Issues  in  Display  Variables 

In  the  past,  specifications  for  visual  simulation  systems  have 
been  incomplete  in  that  they  have  not  contained  parameters  and  meas¬ 
urement  procedures  pertinent  to  the  observer's  visual  requirements. 

For  example,  specification  of  video  bandwidth,  line  rate,  and  maximum 
video  voltage  may  define  the  limiting  performance  of  the  display,  but 
they  do  not  define  the  luminance  range,  the  MTF,  the  actual  limiting 
resolution,  the  line  pairing  or  interlace  accuracy,  or  other  vision- 
related  characteristics.  What  is  needed,  of  course,  is  a  set  of  meas¬ 
urable  displayed  image  variables,  the  required  levels  of  which  are 
minimum  for  adequate  visual  performance  of  the  pilot.  In  addition, 
there  must  be  specified  the  operational  procedures  by  which  quality 
control  of  these  characteristics  is  maintained  in  both  acceptance 
testing  and  subsequent  maintenance.  To  date,  only  a  few  military 
operational  systems  have  been  purchased  and  maintained  under  these 
procedures,  and  no  simulator  systems  have  had  to  meet  stringent  and 
pertinent  design  requirements. 

The  following  paragraphs  describe  but  four  of  these  measurement 
categories.  While  others  exist,  it  is  the  proper  province  of  the 
system  designer  to  specify  both  the  critical  design  parameters  and 
the  procedures  by  which  those  parameters  will  be  measured  and  main¬ 
tained.  The  following  is  therefore  suggestive,  and  not  definitive. 


Radiometric/Photometric  Control 

Many  CRT  and  flat  panel  displays  are  stated  to  provide  certain 
levels  of  luminance  (albeit  often  called  "brightness")  under  maximum 
signal  conditions.  Some  contain  specified  luminance  levels  for  zero 
voltage  input.  Unfortunately,  two  problems  exist  in  these  specifica¬ 
tions.  First,  the  maximum  luminance  level  is  not  achieved  in  a  CRT 
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without  some  loss  of  resolution  due  to  spot  bloom;  second,  the  "black" 
level  is  never  at  zero  luminance  due  to  scattering,  faceplate  reflect¬ 
ance,  and  other  conditions.  Thus,  the  obtained  maximum  modulation  is 
always  less  than  unity,  and  the  true  amount  of  achievable  modulation 
is  never  stated. 

While  the  problem  is  somewhat  different  for  a  solid  state  dis¬ 
play,  the  maximum  luminance  is  often  dependent  on  the  number  of 
pixels  turned  "on"  and  may  also  vary  with  "half-select"  voltage  char¬ 
acteristics  of  the  device,  an  issue  rarely  addressed  in  the  perfor¬ 
mance  specifications  of  the  device. 

Recent  research  has  resulted  in  recommendations  for  radiometric 
scanning  of  the  display  device  under  known  input  conditions.  A 
scanning  monochromator  is  used  to  obtain  radiant  energy  per  unit 
wavelength,  which  is  then  converted  to  photometric  power  and  ulti¬ 
mately  to  luminance  if  that  is  of  interest  to  the  user.  Accuracies 
are  obtainable  of  better  than  1  percent,  and  dominant  wavelength  can 
be  measured  to  within  2  nanometers.  CIE  coordinates  can  likewise  be 
calculated  from  the  radiometric  scans  to  within  .002  in  x  and  y  (Far¬ 
ley  and  Gutmann,  1980)  . 

These  measurement  techniques  are  particularly  valuable  in  color 
research  and  in  chromatic  control  of  the  display  when  constant  lumi¬ 
nance  or  constant  brightness  is  needed.  Further,  computerized  scanning 
and  calibration  is  achievable  on  a  scheduled  basis,  using  self¬ 
calibrating  radiometric  equipment  and  programmed  inputs  to  the  display 
device.  Outputs  from  periodic  calibration  can  be  used  to  define  re¬ 
quirements  for  adjustment,  maintenance,  or  replacement  of  display 
components.  In  particular,  since  the  luminance  and  chromatic  drift 
of  many  CRTs  is  as  much  as  10  percent  in  less  than  one  hour,  periodic 
measurement  is  needed  during  research  activities  and  perhaps  during 
simulation  tests. 


Measuring  MTF 

The  static  MTF  of  a  display  can  most  easily  be  measured  with 
either  a  square-wave  or  a  sine-wave  grating  inputted  to  the  display, 
and  the  display  surface  scanned  microphotometrically  or  microradio- 
metrically  with  a  slit  scanning  microscope.  The  scan,  taken  over 
several  cycles,  is  then  Fourier  analyzed  to  determine  (1)  the  modu¬ 
lation  at  the  fundamental  frequency,  (2)  the  spatial  frequency  of 
the  fundamental,  to  check  for  magnification  errors,  and  (3)  any  har¬ 
monic  distortion  as  evidenced  by  inappropriate  luminous  power  in 
higher  harmonics.  Since  digitally  addressed  displays  cannot  reli¬ 
ably  and  accurately  reproduce  a  sine  wave  at  high  spatial  frequencies, 
the  most  convenient  technique  is  to  input  a  square  wave,  followed  by 
correction  by  a  factor  of  4/pi  to  obtain  the  equivalent  sine-wave 
modulation,  assuming  system  linearity. 

Systematic  inputs  of  varying  spatial  frequencies,  followed  by 
plotting  of  the  modulation  in  the  Fourier  spatial  fundamental,  yields 
the  MTF  curve.  This  procedure  should  be  done  in  both  the  vertical 
and  the  horizontal  display  dimensions  to  measure  both  MTSs,  as  most 
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displays  are  anisotropic.  In  addition,  when  a  raster  scan  is  used, 
the  luminous  power  in  the  raster  can  be  determined  in  this  fashion 
with  a  constant  (gray)  image  input  signal.  If  any  line  pairing  ex¬ 
ists  in  the  raster,  the  Fourier  line  spectrum  will  show  a  reduced 
modulation  at  the  raster  fundamental  frequency  and  more  modulation 
at  higher  order  harmonics.  These  and  other  measurement  techniques 
are  described  in  Snyder  et  al.  (1974)  and  in  Snyder  and  Maddox 
(1978) .  Dynamic  MTFs  are  more  complex  and  are  critical  to  this 
problem,  but  need  not  be  discussed  here. 


Measuring  Jitter 

The  measurement  of  jitter  requires  careful  measurement  ir  the 
temporal  rather  than  in  the  spatial  domain.  Jitter  is  most  easily 
measured  by  placing  the  object  plane  aperture  of  the  microphotometer 
at  the  edge  of  a  raster  or  image  element  line,  and  then  sampling  the 
radiance  or  luminance  output  during  successive  refresh  cycles.  Vari¬ 
ation  of  this  output,  for  a  constant  input,  denotes  jitter  (or  lumi¬ 
nous  noise) .  Zero-lag  sampling  can  be  most  easily  obtained  by  record¬ 
ing  directly  from  the  dynode  of  a  photomultiplier  tube  photometer  and 
presenting  successive  output  levels  on  a  storage  oscilloscope. 
Measurements  in  rms  luminance  can  be  made  directly  from  the  oscillo¬ 
scope  image  if  the  photomultiplier  tube  output  has  been  calibrated. 
Details  of  this  technique  are  described  in  Snyder  et  al.  (1979). 


Dynamic  Characteristics 

CRT  displays  are  subject  to  several  forms  of  dynamic  changes 
over  time.  Noted  above  were  jitter,  luminance  instability,  and 
chromatic  instability.  These  changes  are  largely  due  to  fluctuations 
in  line  voltage,  heating  of  electronic  elements  within  the  display, 
and  fluctuations  in  driving  signals.  Other  image-related,  dynamic 
characteristics  of  concern  include  chrominance  changes  due  to  mis- 
convergence  of  the  color  electron  guns  with  the  shadow  mask,  long¬ 
term  drifts  in  luminance  and  therefore  MTF,  and  long-term  changes  in 
spot  size,  which  also  influence  the  MTF  and  limiting  resolution. 

Each  of  these  dynamic  changes  can  be  measured  with  standardized 
spatial  or  temporal  microphotometric  or  microradiometric  measurements, 
usually  of  the  form  described  above.  What  is  critical  is  that  they 
are  measured  and  are  not  simply  assumed  to  be  stable  for  the  useful 
duration  of  the  system.  Since  such  degradations  are  gradual  (often 
called  "graceful"),  they  tend  to  go  unnoticed,  all  the  time  poten¬ 
tially  degrading  performance  of  the  user  until  total  failure  occurs. 

Measurement  and  drift  are  especially  critical  for  those  param¬ 
eters  shown  to  be  sensitively  related  to  pilot  performance.  During 
research  studies  to  determine  display  requirements,  in  particular, 
these  measurements  should  be  made  systematically  and  periodically, 
recorded,  and  related  to  any  trends  in  subject  performance.  In  an 
operational  training  environment,  such  measurements  would  similarly 
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be  madf,  recorded,  and  related  to  long-term  trends  in  training  effec¬ 
tiveness.  To  the  extent  that  one  must  be  careful  in  specifying  opti¬ 
mum  levels  of  display  system  parameters,  one  must  also  be  careful  to 
retain  those  levels  during  operational  usage. 


Issues  Warranting  Research  Consideration 


Many  variables  relating  to  image  content  may  require  careful  re¬ 
search  to  yield  flight  simulation  displays  adequate  for  efficient 
low-level  flight  training.  However,  specification  of  the  image  con¬ 
tent  variables  will  be  of  little  value  if  the  displayed  image  is  in¬ 
adequate  to  present  the  content  with  acceptable  fidelity,  consistency, 
and  quality.  For  these  reasons,  the  following  variables  may  be 
critical  in  display  specification/design  and  warrant  experimental  or 
analytical  investigation  as  to  their  impact  on  performance  and  train¬ 
ing  transfer  for  low-level  flight: 

1.  Field  of  View:  The  relationship  between  FOV  (vertical  and 
horizontal)  and  pilot  performance  for  training  effectiveness 
in  low-altitude  simulated  flight  must  be  determined,  as  this 
is  critical  to  the  design  of  a  variety  of  display  system 
characteristics . 

2.  MTF :  The  effect  of  system  MTF  on  pilot  performance  for 
training  effectiveness  must  be  similarly  measured.  In¬ 
creases  in  MTF  are  difficult  and  expensive,  and  overspeci¬ 
fication  is  unfortunately  possible,  though  greatly 
uneconomical. 

3.  Virtual  versus  Real  Image:  A  direct  comparison  of  collimated 
with  noncollimated  images  should  be  made,  using  pilot  train¬ 
ing  performance  as  well  as  subjective  ratings  as  criterion 
measures.  Size,  complexity,  and  cost  considerations  relate 
directly  to  this  evaluation. 

4.  Jitter:  Some  jitter  (or  other  dynamic  sources  of  image 
quality  loss)  will  always  remain  in  CRT -based  systems.  The 
key  issue  is  the  degree  to  which  jitter  must  be  controlled 
through  expensive  and  complex  design.  No  data  currently 
exist,  and  empirical  studies  are  needed  to  relate  jitter 
amplitude  and  frequency  to  pilot  performance. 

These  research  recommendations  are  considered  to  be  compatible 
with  the  image  content  recommendations  of  appendixes  A  through  D. 
Determination  of  image  content  for  pilot  interpretation  without  ade¬ 
quate  attention  to  display  quality  parameters  will  not  result  in  any 
significant  capability  improvement.  Similarly,  improvements  in  dis¬ 
play  quality,  without  regard  for  image  content  considerations  will 
yield  little  gain.  A  compatible,  integrated  research  effort  that 
takes  into  account  the  perceptual  content  requirements  of  the  trainee 
as  well  as  the  image  quality  variables  is  therefore  seen  as  most 
promising. 
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APPENDIX  F 
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COMMITTEE  ON  VISION 
NATIONAL  RESEARCH  COUNCIL 
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of  Visual  Cues 
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D.  Fender 
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Attentional  Factors 
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